Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=1),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1300,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.472861
Epoch 1
Validation binary_cross_entropy = 0.473645
Epoch 2
Validation binary_cross_entropy = 1.413935
Epoch 3
Validation binary_cross_entropy = 0.413583
Epoch 4
Loss = 8.0778e-01, PNorm = 68.1502, GNorm = 21.4564, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.562545
Epoch 5
Validation binary_cross_entropy = 1.162232
Epoch 6
Validation binary_cross_entropy = 0.648112
Epoch 7
Validation binary_cross_entropy = 0.773798
Epoch 8
Validation binary_cross_entropy = 1.162189
Epoch 9
Loss = 3.6959e-01, PNorm = 68.3166, GNorm = 9.9284, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.641350
Epoch 10
Validation binary_cross_entropy = 0.592651
Epoch 11
Validation binary_cross_entropy = 0.731764
Epoch 12
Validation binary_cross_entropy = 0.605069
Epoch 13
Validation binary_cross_entropy = 0.677397
Epoch 14
Loss = 3.7527e-01, PNorm = 68.5033, GNorm = 4.7548, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.892322
Epoch 15
Validation binary_cross_entropy = 0.759294
Epoch 16
Validation binary_cross_entropy = 0.736556
Epoch 17
Validation binary_cross_entropy = 0.629282
Epoch 18
Validation binary_cross_entropy = 0.757352
Epoch 19
Loss = 3.5128e-01, PNorm = 68.6403, GNorm = 4.2021, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.575861
Epoch 20
Validation binary_cross_entropy = 0.571229
Epoch 21
Validation binary_cross_entropy = 0.598973
Epoch 22
Validation binary_cross_entropy = 0.629254
Epoch 23
Validation binary_cross_entropy = 0.647712
Epoch 24
Loss = 5.4776e-02, PNorm = 68.7354, GNorm = 1.4358, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.678190
Epoch 25
Validation binary_cross_entropy = 0.589898
Epoch 26
Validation binary_cross_entropy = 0.670148
Epoch 27
Validation binary_cross_entropy = 0.543351
Epoch 28
Validation binary_cross_entropy = 0.529938
Epoch 29
Loss = 9.3610e-02, PNorm = 68.8185, GNorm = 0.6616, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.503764
Epoch 30
Validation binary_cross_entropy = 0.545919
Epoch 31
Validation binary_cross_entropy = 0.501411
Epoch 32
Validation binary_cross_entropy = 0.525164
Epoch 33
Validation binary_cross_entropy = 0.544317
Epoch 34
Loss = 9.9842e-02, PNorm = 68.8971, GNorm = 4.0844, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.566414
Epoch 35
Validation binary_cross_entropy = 0.580386
Epoch 36
Validation binary_cross_entropy = 0.539742
Epoch 37
Validation binary_cross_entropy = 0.545914
Epoch 38
Validation binary_cross_entropy = 0.557376
Epoch 39
Loss = 1.4670e-01, PNorm = 68.9845, GNorm = 6.6660, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.580235
Epoch 40
Validation binary_cross_entropy = 0.673056
Epoch 41
Validation binary_cross_entropy = 0.589102
Epoch 42
Validation binary_cross_entropy = 0.679634
Epoch 43
Validation binary_cross_entropy = 0.610195
Epoch 44
Loss = 3.8979e-02, PNorm = 69.0799, GNorm = 1.1842, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.793964
Epoch 45
Validation binary_cross_entropy = 0.666211
Epoch 46
Validation binary_cross_entropy = 0.773154
Epoch 47
Validation binary_cross_entropy = 0.695581
Epoch 48
Validation binary_cross_entropy = 0.737840
Epoch 49
Loss = 8.9838e-02, PNorm = 69.1747, GNorm = 3.7679, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.767814
Epoch 50
Validation binary_cross_entropy = 0.673159
Epoch 51
Validation binary_cross_entropy = 0.783410
Epoch 52
Validation binary_cross_entropy = 0.691545
Epoch 53
Validation binary_cross_entropy = 0.890525
Epoch 54
Loss = 2.5801e-01, PNorm = 69.2594, GNorm = 5.7470, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.699434
Epoch 55
Validation binary_cross_entropy = 0.668431
Epoch 56
Validation binary_cross_entropy = 0.678716
Epoch 57
Validation binary_cross_entropy = 0.637791
Epoch 58
Validation binary_cross_entropy = 0.617201
Epoch 59
Loss = 3.0691e-02, PNorm = 69.3795, GNorm = 1.8286, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.648596
Epoch 60
Validation binary_cross_entropy = 0.770292
Epoch 61
Validation binary_cross_entropy = 0.720992
Epoch 62
Validation binary_cross_entropy = 0.706986
Epoch 63
Validation binary_cross_entropy = 0.736931
Epoch 64
Loss = 4.0101e-02, PNorm = 69.4716, GNorm = 2.5570, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.688433
Epoch 65
Validation binary_cross_entropy = 0.654316
Epoch 66
Validation binary_cross_entropy = 0.632827
Epoch 67
Validation binary_cross_entropy = 0.629665
Epoch 68
Validation binary_cross_entropy = 0.642176
Epoch 69
Loss = 2.5191e-02, PNorm = 69.5341, GNorm = 0.5065, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.665679
Epoch 70
Validation binary_cross_entropy = 0.677748
Epoch 71
Validation binary_cross_entropy = 0.669862
Epoch 72
Validation binary_cross_entropy = 0.676825
Epoch 73
Validation binary_cross_entropy = 0.686038
Epoch 74
Loss = 3.0170e-02, PNorm = 69.5763, GNorm = 0.2815, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.708359
Epoch 75
Validation binary_cross_entropy = 0.721787
Epoch 76
Validation binary_cross_entropy = 0.714009
Epoch 77
Validation binary_cross_entropy = 0.682589
Epoch 78
Validation binary_cross_entropy = 0.682572
Epoch 79
Loss = 4.6442e-02, PNorm = 69.6101, GNorm = 2.3296, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.680723
Epoch 80
Validation binary_cross_entropy = 0.706168
Epoch 81
Validation binary_cross_entropy = 0.744203
Epoch 82
Validation binary_cross_entropy = 0.755096
Epoch 83
Validation binary_cross_entropy = 0.748682
Epoch 84
Loss = 7.2217e-03, PNorm = 69.6360, GNorm = 0.2038, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.748146
Epoch 85
Validation binary_cross_entropy = 0.766121
Epoch 86
Validation binary_cross_entropy = 0.773846
Epoch 87
Validation binary_cross_entropy = 0.789476
Epoch 88
Validation binary_cross_entropy = 0.795501
Epoch 89
Loss = 4.4289e-02, PNorm = 69.6618, GNorm = 0.6316, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.752101
Epoch 90
Validation binary_cross_entropy = 0.834657
Epoch 91
Validation binary_cross_entropy = 0.932058
Epoch 92
Validation binary_cross_entropy = 0.794714
Epoch 93
Validation binary_cross_entropy = 0.764478
Epoch 94
Loss = 7.3986e-02, PNorm = 69.7084, GNorm = 2.4440, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.965317
Epoch 95
Validation binary_cross_entropy = 0.740977
Epoch 96
Validation binary_cross_entropy = 0.936625
Epoch 97
Validation binary_cross_entropy = 0.989791
Epoch 98
Validation binary_cross_entropy = 0.682031
Epoch 99
Loss = 2.1772e-02, PNorm = 69.7961, GNorm = 0.4735, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.699556
Epoch 100
Validation binary_cross_entropy = 0.754612
Epoch 101
Validation binary_cross_entropy = 0.708937
Epoch 102
Validation binary_cross_entropy = 0.784737
Epoch 103
Validation binary_cross_entropy = 0.922469
Epoch 104
Loss = 5.5462e-02, PNorm = 69.9074, GNorm = 3.0032, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.968282
Epoch 105
Validation binary_cross_entropy = 0.879609
Epoch 106
Validation binary_cross_entropy = 0.845317
Epoch 107
Validation binary_cross_entropy = 0.848422
Epoch 108
Validation binary_cross_entropy = 0.850741
Epoch 109
Loss = 6.2469e-02, PNorm = 69.9785, GNorm = 3.1573, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.894984
Epoch 110
Validation binary_cross_entropy = 1.018100
Epoch 111
Validation binary_cross_entropy = 0.972102
Epoch 112
Validation binary_cross_entropy = 0.855690
Epoch 113
Validation binary_cross_entropy = 0.916511
Epoch 114
Loss = 1.3244e-01, PNorm = 70.0428, GNorm = 1.5486, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.882366
Epoch 115
Validation binary_cross_entropy = 0.898312
Epoch 116
Validation binary_cross_entropy = 0.954598
Epoch 117
Validation binary_cross_entropy = 0.958659
Epoch 118
Validation binary_cross_entropy = 0.813575
Epoch 119
Loss = 2.4432e-02, PNorm = 70.1084, GNorm = 0.1213, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.720184
Epoch 120
Validation binary_cross_entropy = 0.760429
Epoch 121
Validation binary_cross_entropy = 0.710951
Epoch 122
Validation binary_cross_entropy = 0.808868
Epoch 123
Validation binary_cross_entropy = 0.895761
Epoch 124
Loss = 2.0988e-02, PNorm = 70.1820, GNorm = 1.4184, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.952723
Epoch 125
Validation binary_cross_entropy = 0.910778
Epoch 126
Validation binary_cross_entropy = 0.835925
Epoch 127
Validation binary_cross_entropy = 0.767195
Epoch 128
Validation binary_cross_entropy = 0.741056
Epoch 129
Loss = 5.8463e-02, PNorm = 70.2559, GNorm = 2.4946, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.762611
Epoch 130
Validation binary_cross_entropy = 0.813883
Epoch 131
Validation binary_cross_entropy = 0.881851
Epoch 132
Validation binary_cross_entropy = 0.888861
Epoch 133
Validation binary_cross_entropy = 0.798741
Epoch 134
Loss = 8.5313e-03, PNorm = 70.3256, GNorm = 0.7142, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.804174
Epoch 135
Validation binary_cross_entropy = 0.884356
Epoch 136
Validation binary_cross_entropy = 0.857100
Epoch 137
Validation binary_cross_entropy = 0.775215
Epoch 138
Validation binary_cross_entropy = 0.842096
Epoch 139
Loss = 3.2482e-02, PNorm = 70.3788, GNorm = 4.1188, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.902732
Epoch 140
Validation binary_cross_entropy = 0.855215
Epoch 141
Validation binary_cross_entropy = 0.825868
Epoch 142
Validation binary_cross_entropy = 0.782913
Epoch 143
Validation binary_cross_entropy = 0.788113
Epoch 144
Loss = 2.5728e-02, PNorm = 70.4184, GNorm = 1.4292, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.856483
Epoch 145
Validation binary_cross_entropy = 0.875552
Epoch 146
Validation binary_cross_entropy = 0.836615
Epoch 147
Validation binary_cross_entropy = 0.800517
Epoch 148
Validation binary_cross_entropy = 0.778811
Epoch 149
Loss = 4.6029e-03, PNorm = 70.4607, GNorm = 0.3871, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.770563
Epoch 150
Validation binary_cross_entropy = 0.781610
Epoch 151
Validation binary_cross_entropy = 0.780153
Epoch 152
Validation binary_cross_entropy = 0.785540
Epoch 153
Validation binary_cross_entropy = 0.795131
Epoch 154
Loss = 1.8693e-02, PNorm = 70.5123, GNorm = 2.6818, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.834002
Epoch 155
Validation binary_cross_entropy = 0.872242
Epoch 156
Validation binary_cross_entropy = 0.846666
Epoch 157
Validation binary_cross_entropy = 0.850289
Epoch 158
Validation binary_cross_entropy = 0.848507
Epoch 159
Loss = 3.8355e-03, PNorm = 70.5502, GNorm = 0.1543, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.849698
Epoch 160
Validation binary_cross_entropy = 0.836397
Epoch 161
Validation binary_cross_entropy = 0.826194
Epoch 162
Validation binary_cross_entropy = 0.818699
Epoch 163
Validation binary_cross_entropy = 0.816663
Epoch 164
Loss = 3.3578e-03, PNorm = 70.5737, GNorm = 0.2067, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.818447
Epoch 165
Validation binary_cross_entropy = 0.820317
Epoch 166
Validation binary_cross_entropy = 0.826202
Epoch 167
Validation binary_cross_entropy = 0.868569
Epoch 168
Validation binary_cross_entropy = 0.898996
Epoch 169
Loss = 4.1159e-03, PNorm = 70.5964, GNorm = 0.1737, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.915396
Epoch 170
Validation binary_cross_entropy = 0.864773
Epoch 171
Validation binary_cross_entropy = 0.838257
Epoch 172
Validation binary_cross_entropy = 0.833328
Epoch 173
Validation binary_cross_entropy = 0.836484
Epoch 174
Loss = 2.2212e-02, PNorm = 70.6245, GNorm = 0.0692, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.869234
Epoch 175
Validation binary_cross_entropy = 0.894626
Epoch 176
Validation binary_cross_entropy = 0.905031
Epoch 177
Validation binary_cross_entropy = 0.899691
Epoch 178
Validation binary_cross_entropy = 0.875569
Epoch 179
Loss = 1.4498e-02, PNorm = 70.6480, GNorm = 2.0938, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.856318
Epoch 180
Validation binary_cross_entropy = 0.849329
Epoch 181
Validation binary_cross_entropy = 0.837223
Epoch 182
Validation binary_cross_entropy = 0.827828
Epoch 183
Validation binary_cross_entropy = 0.818115
Epoch 184
Loss = 1.9251e-03, PNorm = 70.6663, GNorm = 0.1023, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.810143
Epoch 185
Validation binary_cross_entropy = 0.808761
Epoch 186
Validation binary_cross_entropy = 0.789547
Epoch 187
Validation binary_cross_entropy = 0.781452
Epoch 188
Validation binary_cross_entropy = 0.782843
Epoch 189
Loss = 9.4704e-03, PNorm = 70.6834, GNorm = 0.2216, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.801541
Epoch 190
Validation binary_cross_entropy = 0.860094
Epoch 191
Validation binary_cross_entropy = 0.898133
Epoch 192
Validation binary_cross_entropy = 0.900458
Epoch 193
Validation binary_cross_entropy = 0.891414
Epoch 194
Loss = 1.1766e-02, PNorm = 70.7071, GNorm = 2.3169, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.865171
Epoch 195
Validation binary_cross_entropy = 0.830959
Epoch 196
Validation binary_cross_entropy = 0.817497
Epoch 197
Validation binary_cross_entropy = 0.833219
Epoch 198
Validation binary_cross_entropy = 0.898945
Epoch 199
Loss = 3.9044e-03, PNorm = 70.7291, GNorm = 0.2161, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.960790
Epoch 200
Validation binary_cross_entropy = 0.981997
Epoch 201
Validation binary_cross_entropy = 0.935312
Epoch 202
Validation binary_cross_entropy = 0.897787
Epoch 203
Validation binary_cross_entropy = 0.869386
Epoch 204
Loss = 6.1174e-03, PNorm = 70.7534, GNorm = 0.0448, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.861278
Epoch 205
Validation binary_cross_entropy = 0.855684
Epoch 206
Validation binary_cross_entropy = 0.853431
Epoch 207
Validation binary_cross_entropy = 0.863005
Epoch 208
Validation binary_cross_entropy = 0.912394
Epoch 209
Loss = 1.8898e-03, PNorm = 70.7846, GNorm = 0.1023, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.969905
Epoch 210
Validation binary_cross_entropy = 1.017252
Epoch 211
Validation binary_cross_entropy = 1.030026
Epoch 212
Validation binary_cross_entropy = 0.972848
Epoch 213
Validation binary_cross_entropy = 0.933358
Epoch 214
Loss = 1.2150e-03, PNorm = 70.8189, GNorm = 0.0391, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.910025
Epoch 215
Validation binary_cross_entropy = 0.900819
Epoch 216
Validation binary_cross_entropy = 0.895187
Epoch 217
Validation binary_cross_entropy = 0.898651
Epoch 218
Validation binary_cross_entropy = 0.904763
Epoch 219
Loss = 8.8819e-03, PNorm = 70.8524, GNorm = 0.0312, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.875425
Epoch 220
Validation binary_cross_entropy = 0.858166
Epoch 221
Validation binary_cross_entropy = 0.878071
Epoch 222
Validation binary_cross_entropy = 0.895879
Epoch 223
Validation binary_cross_entropy = 0.910035
Epoch 224
Loss = 8.5390e-03, PNorm = 70.8855, GNorm = 0.5059, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.920188
Epoch 225
Validation binary_cross_entropy = 0.953534
Epoch 226
Validation binary_cross_entropy = 0.964182
Epoch 227
Validation binary_cross_entropy = 0.926654
Epoch 228
Validation binary_cross_entropy = 0.895810
Epoch 229
Loss = 1.6330e-03, PNorm = 70.9109, GNorm = 0.0345, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.875267
Epoch 230
Validation binary_cross_entropy = 0.861993
Epoch 231
Validation binary_cross_entropy = 0.861695
Epoch 232
Validation binary_cross_entropy = 0.867446
Epoch 233
Validation binary_cross_entropy = 0.886693
Epoch 234
Loss = 3.7438e-02, PNorm = 70.9394, GNorm = 3.3908, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.911868
Epoch 235
Validation binary_cross_entropy = 0.913777
Epoch 236
Validation binary_cross_entropy = 0.924637
Epoch 237
Validation binary_cross_entropy = 0.937310
Epoch 238
Validation binary_cross_entropy = 1.012255
Epoch 239
Loss = 1.6158e-02, PNorm = 70.9999, GNorm = 0.8959, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.054116
Epoch 240
Validation binary_cross_entropy = 1.028976
Epoch 241
Validation binary_cross_entropy = 0.986051
Epoch 242
Validation binary_cross_entropy = 0.919634
Epoch 243
Validation binary_cross_entropy = 0.852044
Epoch 244
Loss = 4.6817e-03, PNorm = 71.0693, GNorm = 0.5344, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.821058
Epoch 245
Validation binary_cross_entropy = 0.821585
Epoch 246
Validation binary_cross_entropy = 0.913084
Epoch 247
Validation binary_cross_entropy = 1.049566
Epoch 248
Validation binary_cross_entropy = 1.103094
Epoch 249
Loss = 1.1439e-02, PNorm = 71.1592, GNorm = 0.9022, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.085032
Epoch 250
Validation binary_cross_entropy = 1.022363
Epoch 251
Validation binary_cross_entropy = 0.942940
Epoch 252
Validation binary_cross_entropy = 0.879870
Epoch 253
Validation binary_cross_entropy = 0.831963
Epoch 254
Loss = 8.5514e-04, PNorm = 71.2515, GNorm = 0.0281, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.805281
Epoch 255
Validation binary_cross_entropy = 0.797231
Epoch 256
Validation binary_cross_entropy = 0.801662
Epoch 257
Validation binary_cross_entropy = 0.833844
Epoch 258
Validation binary_cross_entropy = 0.883469
Epoch 259
Loss = 4.2492e-03, PNorm = 71.3131, GNorm = 0.2297, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.936103
Epoch 260
Validation binary_cross_entropy = 1.011836
Epoch 261
Validation binary_cross_entropy = 1.051978
Epoch 262
Validation binary_cross_entropy = 1.021890
Epoch 263
Validation binary_cross_entropy = 0.988711
Epoch 264
Loss = 3.9450e-02, PNorm = 71.3663, GNorm = 3.6437, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.927329
Epoch 265
Validation binary_cross_entropy = 0.864456
Epoch 266
Validation binary_cross_entropy = 0.847491
Epoch 267
Validation binary_cross_entropy = 0.863489
Epoch 268
Validation binary_cross_entropy = 0.897818
Epoch 269
Loss = 1.7771e-03, PNorm = 71.4243, GNorm = 0.1001, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.950638
Epoch 270
Validation binary_cross_entropy = 0.996951
Epoch 271
Validation binary_cross_entropy = 1.036158
Epoch 272
Validation binary_cross_entropy = 1.043620
Epoch 273
Validation binary_cross_entropy = 1.022506
Epoch 274
Loss = 1.4431e-03, PNorm = 71.4782, GNorm = 0.0789, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.991986
Epoch 275
Validation binary_cross_entropy = 1.009169
Epoch 276
Validation binary_cross_entropy = 1.019378
Epoch 277
Validation binary_cross_entropy = 1.022446
Epoch 278
Validation binary_cross_entropy = 0.994723
Epoch 279
Loss = 4.9238e-03, PNorm = 71.5129, GNorm = 0.0367, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.958213
Epoch 280
Validation binary_cross_entropy = 0.936692
Epoch 281
Validation binary_cross_entropy = 0.926035
Epoch 282
Validation binary_cross_entropy = 0.917717
Epoch 283
Validation binary_cross_entropy = 0.909644
Epoch 284
Loss = 2.3048e-02, PNorm = 71.5300, GNorm = 0.3417, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.941421
Epoch 285
Validation binary_cross_entropy = 1.003683
Epoch 286
Validation binary_cross_entropy = 1.039122
Epoch 287
Validation binary_cross_entropy = 1.034154
Epoch 288
Validation binary_cross_entropy = 1.017899
Epoch 289
Loss = 1.0557e-03, PNorm = 71.5520, GNorm = 0.0286, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.003529
Epoch 290
Validation binary_cross_entropy = 0.990216
Epoch 291
Validation binary_cross_entropy = 0.978514
Epoch 292
Validation binary_cross_entropy = 0.968690
Epoch 293
Validation binary_cross_entropy = 1.007712
Epoch 294
Loss = 3.0518e-03, PNorm = 71.5881, GNorm = 0.0655, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.045751
Epoch 295
Validation binary_cross_entropy = 1.046932
Epoch 296
Validation binary_cross_entropy = 1.026786
Epoch 297
Validation binary_cross_entropy = 1.003848
Epoch 298
Validation binary_cross_entropy = 0.972443
Epoch 299
Loss = 3.2763e-03, PNorm = 71.6279, GNorm = 0.2544, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.955682
Model 0 best validation binary_cross_entropy = 0.413583 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.202354
Ensemble test binary_cross_entropy = 0.202354
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.714586
Epoch 1
Validation binary_cross_entropy = 0.336724
Epoch 2
Validation binary_cross_entropy = 0.879957
Epoch 3
Validation binary_cross_entropy = 0.403319
Epoch 4
Loss = 4.7637e-01, PNorm = 68.1462, GNorm = 9.2321, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.463373
Epoch 5
Validation binary_cross_entropy = 0.510878
Epoch 6
Validation binary_cross_entropy = 0.487807
Epoch 7
Validation binary_cross_entropy = 0.737735
Epoch 8
Validation binary_cross_entropy = 0.591702
Epoch 9
Loss = 2.7458e-01, PNorm = 68.2929, GNorm = 7.1698, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.631283
Epoch 10
Validation binary_cross_entropy = 0.785159
Epoch 11
Validation binary_cross_entropy = 0.598545
Epoch 12
Validation binary_cross_entropy = 0.622498
Epoch 13
Validation binary_cross_entropy = 0.624287
Epoch 14
Loss = 4.4603e-01, PNorm = 68.4459, GNorm = 16.3298, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.705973
Epoch 15
Validation binary_cross_entropy = 1.027878
Epoch 16
Validation binary_cross_entropy = 0.628155
Epoch 17
Validation binary_cross_entropy = 0.718813
Epoch 18
Validation binary_cross_entropy = 0.570116
Epoch 19
Loss = 1.4272e-01, PNorm = 68.5656, GNorm = 7.4246, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.917152
Epoch 20
Validation binary_cross_entropy = 0.527804
Epoch 21
Validation binary_cross_entropy = 0.649181
Epoch 22
Validation binary_cross_entropy = 0.550509
Epoch 23
Validation binary_cross_entropy = 0.705383
Epoch 24
Loss = 2.1968e-01, PNorm = 68.6739, GNorm = 4.2287, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.653683
Epoch 25
Validation binary_cross_entropy = 0.742543
Epoch 26
Validation binary_cross_entropy = 0.701198
Epoch 27
Validation binary_cross_entropy = 0.682491
Epoch 28
Validation binary_cross_entropy = 0.728439
Epoch 29
Loss = 3.4329e-01, PNorm = 68.7650, GNorm = 4.9423, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.655231
Epoch 30
Validation binary_cross_entropy = 0.676102
Epoch 31
Validation binary_cross_entropy = 0.575668
Epoch 32
Validation binary_cross_entropy = 0.637043
Epoch 33
Validation binary_cross_entropy = 0.661674
Epoch 34
Loss = 8.4822e-02, PNorm = 68.8584, GNorm = 3.1403, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.678990
Epoch 35
Validation binary_cross_entropy = 0.687906
Epoch 36
Validation binary_cross_entropy = 0.690747
Epoch 37
Validation binary_cross_entropy = 0.691266
Epoch 38
Validation binary_cross_entropy = 0.713396
Epoch 39
Loss = 2.5686e-01, PNorm = 68.9542, GNorm = 8.1590, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.748166
Epoch 40
Validation binary_cross_entropy = 0.664095
Epoch 41
Validation binary_cross_entropy = 0.644708
Epoch 42
Validation binary_cross_entropy = 0.653796
Epoch 43
Validation binary_cross_entropy = 0.644542
Epoch 44
Loss = 4.4727e-02, PNorm = 69.0713, GNorm = 1.7317, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.665110
Epoch 45
Validation binary_cross_entropy = 0.693147
Epoch 46
Validation binary_cross_entropy = 0.740103
Epoch 47
Validation binary_cross_entropy = 0.744895
Epoch 48
Validation binary_cross_entropy = 0.766520
Epoch 49
Loss = 3.7112e-02, PNorm = 69.1363, GNorm = 1.8163, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.723552
Epoch 50
Validation binary_cross_entropy = 0.660292
Epoch 51
Validation binary_cross_entropy = 0.635399
Epoch 52
Validation binary_cross_entropy = 0.631776
Epoch 53
Validation binary_cross_entropy = 0.732325
Epoch 54
Loss = 4.8970e-02, PNorm = 69.1970, GNorm = 2.8553, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.748250
Epoch 55
Validation binary_cross_entropy = 0.665418
Epoch 56
Validation binary_cross_entropy = 0.625050
Epoch 57
Validation binary_cross_entropy = 0.604371
Epoch 58
Validation binary_cross_entropy = 0.613935
Epoch 59
Loss = 5.9996e-02, PNorm = 69.3032, GNorm = 1.4177, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.666512
Epoch 60
Validation binary_cross_entropy = 0.700025
Epoch 61
Validation binary_cross_entropy = 0.694648
Epoch 62
Validation binary_cross_entropy = 0.677321
Epoch 63
Validation binary_cross_entropy = 0.721607
Epoch 64
Loss = 3.2764e-02, PNorm = 69.3712, GNorm = 1.1418, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.801884
Epoch 65
Validation binary_cross_entropy = 0.826932
Epoch 66
Validation binary_cross_entropy = 0.776465
Epoch 67
Validation binary_cross_entropy = 0.803272
Epoch 68
Validation binary_cross_entropy = 0.742954
Epoch 69
Loss = 2.9789e-02, PNorm = 69.4300, GNorm = 0.2827, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.714972
Epoch 70
Validation binary_cross_entropy = 0.795595
Epoch 71
Validation binary_cross_entropy = 0.744471
Epoch 72
Validation binary_cross_entropy = 0.640926
Epoch 73
Validation binary_cross_entropy = 0.664349
Epoch 74
Loss = 9.9079e-02, PNorm = 69.4888, GNorm = 1.2652, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.711702
Epoch 75
Validation binary_cross_entropy = 0.802586
Epoch 76
Validation binary_cross_entropy = 0.782533
Epoch 77
Validation binary_cross_entropy = 0.699947
Epoch 78
Validation binary_cross_entropy = 0.693258
Epoch 79
Loss = 3.6757e-02, PNorm = 69.5594, GNorm = 2.4434, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.699741
Epoch 80
Validation binary_cross_entropy = 0.732017
Epoch 81
Validation binary_cross_entropy = 0.787575
Epoch 82
Validation binary_cross_entropy = 0.755997
Epoch 83
Validation binary_cross_entropy = 0.687404
Epoch 84
Loss = 9.8064e-02, PNorm = 69.6535, GNorm = 1.7587, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.708852
Epoch 85
Validation binary_cross_entropy = 0.829158
Epoch 86
Validation binary_cross_entropy = 0.832977
Epoch 87
Validation binary_cross_entropy = 0.699170
Epoch 88
Validation binary_cross_entropy = 0.691554
Epoch 89
Loss = 9.8917e-02, PNorm = 69.7256, GNorm = 4.0983, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.744642
Epoch 90
Validation binary_cross_entropy = 0.835842
Epoch 91
Validation binary_cross_entropy = 0.836118
Epoch 92
Validation binary_cross_entropy = 0.805445
Epoch 93
Validation binary_cross_entropy = 0.800231
Epoch 94
Loss = 1.7981e-02, PNorm = 69.7819, GNorm = 1.4075, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.821779
Epoch 95
Validation binary_cross_entropy = 0.834187
Epoch 96
Validation binary_cross_entropy = 0.892042
Epoch 97
Validation binary_cross_entropy = 0.933208
Epoch 98
Validation binary_cross_entropy = 0.821248
Epoch 99
Loss = 1.9350e-02, PNorm = 69.8340, GNorm = 0.3399, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.746628
Epoch 100
Validation binary_cross_entropy = 0.706318
Epoch 101
Validation binary_cross_entropy = 0.703992
Epoch 102
Validation binary_cross_entropy = 0.700591
Epoch 103
Validation binary_cross_entropy = 0.714980
Epoch 104
Loss = 2.6939e-02, PNorm = 69.8780, GNorm = 0.3808, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.764415
Epoch 105
Validation binary_cross_entropy = 0.809536
Epoch 106
Validation binary_cross_entropy = 0.830313
Epoch 107
Validation binary_cross_entropy = 0.874950
Epoch 108
Validation binary_cross_entropy = 0.886951
Epoch 109
Loss = 1.6076e-02, PNorm = 69.9167, GNorm = 0.4634, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.831537
Epoch 110
Validation binary_cross_entropy = 0.792937
Epoch 111
Validation binary_cross_entropy = 0.749322
Epoch 112
Validation binary_cross_entropy = 0.733041
Epoch 113
Validation binary_cross_entropy = 0.749172
Epoch 114
Loss = 3.8863e-02, PNorm = 69.9499, GNorm = 1.6993, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.870036
Epoch 115
Validation binary_cross_entropy = 0.983116
Epoch 116
Validation binary_cross_entropy = 0.919081
Epoch 117
Validation binary_cross_entropy = 0.761897
Epoch 118
Validation binary_cross_entropy = 0.766696
Epoch 119
Loss = 4.2317e-02, PNorm = 69.9885, GNorm = 3.7440, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.831396
Epoch 120
Validation binary_cross_entropy = 0.933530
Epoch 121
Validation binary_cross_entropy = 0.910165
Epoch 122
Validation binary_cross_entropy = 0.761082
Epoch 123
Validation binary_cross_entropy = 0.739005
Epoch 124
Loss = 2.0162e-01, PNorm = 70.0712, GNorm = 5.1774, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.817374
Epoch 125
Validation binary_cross_entropy = 0.901447
Epoch 126
Validation binary_cross_entropy = 0.781863
Epoch 127
Validation binary_cross_entropy = 0.728462
Epoch 128
Validation binary_cross_entropy = 0.802693
Epoch 129
Loss = 1.1070e-01, PNorm = 70.1802, GNorm = 3.9551, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.768965
Epoch 130
Validation binary_cross_entropy = 0.820327
Epoch 131
Validation binary_cross_entropy = 0.863078
Epoch 132
Validation binary_cross_entropy = 0.863538
Epoch 133
Validation binary_cross_entropy = 0.870713
Epoch 134
Loss = 3.6798e-02, PNorm = 70.2708, GNorm = 2.6376, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.839393
Epoch 135
Validation binary_cross_entropy = 0.753854
Epoch 136
Validation binary_cross_entropy = 0.738353
Epoch 137
Validation binary_cross_entropy = 0.736891
Epoch 138
Validation binary_cross_entropy = 0.764660
Epoch 139
Loss = 2.2200e-02, PNorm = 70.3348, GNorm = 0.8475, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.829379
Epoch 140
Validation binary_cross_entropy = 0.732628
Epoch 141
Validation binary_cross_entropy = 0.728642
Epoch 142
Validation binary_cross_entropy = 0.747951
Epoch 143
Validation binary_cross_entropy = 0.743451
Epoch 144
Loss = 4.3721e-02, PNorm = 70.4930, GNorm = 2.0159, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.823767
Epoch 145
Validation binary_cross_entropy = 0.972993
Epoch 146
Validation binary_cross_entropy = 0.939706
Epoch 147
Validation binary_cross_entropy = 0.826830
Epoch 148
Validation binary_cross_entropy = 0.792688
Epoch 149
Loss = 7.0317e-02, PNorm = 70.6056, GNorm = 5.6898, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.896566
Epoch 150
Validation binary_cross_entropy = 0.907073
Epoch 151
Validation binary_cross_entropy = 0.899853
Epoch 152
Validation binary_cross_entropy = 0.959729
Epoch 153
Validation binary_cross_entropy = 0.964516
Epoch 154
Loss = 1.5699e-02, PNorm = 70.6939, GNorm = 0.2077, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.903619
Epoch 155
Validation binary_cross_entropy = 0.873067
Epoch 156
Validation binary_cross_entropy = 0.865500
Epoch 157
Validation binary_cross_entropy = 0.877931
Epoch 158
Validation binary_cross_entropy = 0.908934
Epoch 159
Loss = 4.7780e-03, PNorm = 70.7830, GNorm = 0.1277, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.959394
Epoch 160
Validation binary_cross_entropy = 0.970711
Epoch 161
Validation binary_cross_entropy = 0.943797
Epoch 162
Validation binary_cross_entropy = 0.921113
Epoch 163
Validation binary_cross_entropy = 0.797429
Epoch 164
Loss = 1.7106e-02, PNorm = 70.8451, GNorm = 2.1076, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.754051
Epoch 165
Validation binary_cross_entropy = 0.745747
Epoch 166
Validation binary_cross_entropy = 0.748038
Epoch 167
Validation binary_cross_entropy = 0.778733
Epoch 168
Validation binary_cross_entropy = 0.900912
Epoch 169
Loss = 6.8275e-03, PNorm = 70.9111, GNorm = 0.5022, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.015013
Epoch 170
Validation binary_cross_entropy = 0.993066
Epoch 171
Validation binary_cross_entropy = 0.932809
Epoch 172
Validation binary_cross_entropy = 0.891175
Epoch 173
Validation binary_cross_entropy = 0.875479
Epoch 174
Loss = 7.3812e-03, PNorm = 70.9691, GNorm = 0.4542, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.871981
Epoch 175
Validation binary_cross_entropy = 0.895570
Epoch 176
Validation binary_cross_entropy = 0.934462
Epoch 177
Validation binary_cross_entropy = 0.963490
Epoch 178
Validation binary_cross_entropy = 0.966793
Epoch 179
Loss = 1.5014e-02, PNorm = 71.0177, GNorm = 0.1748, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.933517
Epoch 180
Validation binary_cross_entropy = 0.926639
Epoch 181
Validation binary_cross_entropy = 0.938883
Epoch 182
Validation binary_cross_entropy = 0.946543
Epoch 183
Validation binary_cross_entropy = 0.915127
Epoch 184
Loss = 1.0959e-02, PNorm = 71.0533, GNorm = 0.8733, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.867669
Epoch 185
Validation binary_cross_entropy = 0.851530
Epoch 186
Validation binary_cross_entropy = 0.859676
Epoch 187
Validation binary_cross_entropy = 0.881249
Epoch 188
Validation binary_cross_entropy = 0.916305
Epoch 189
Loss = 8.3448e-03, PNorm = 71.0842, GNorm = 0.3334, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.933225
Epoch 190
Validation binary_cross_entropy = 0.954817
Epoch 191
Validation binary_cross_entropy = 0.946939
Epoch 192
Validation binary_cross_entropy = 0.917144
Epoch 193
Validation binary_cross_entropy = 0.856654
Epoch 194
Loss = 1.8080e-03, PNorm = 71.1241, GNorm = 0.0703, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.819113
Epoch 195
Validation binary_cross_entropy = 0.802867
Epoch 196
Validation binary_cross_entropy = 0.806918
Epoch 197
Validation binary_cross_entropy = 0.856445
Epoch 198
Validation binary_cross_entropy = 0.933227
Epoch 199
Loss = 9.6733e-03, PNorm = 71.1443, GNorm = 1.2661, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.983554
Epoch 200
Validation binary_cross_entropy = 1.010222
Epoch 201
Validation binary_cross_entropy = 1.025423
Epoch 202
Validation binary_cross_entropy = 0.987143
Epoch 203
Validation binary_cross_entropy = 0.947501
Epoch 204
Loss = 2.4982e-03, PNorm = 71.1624, GNorm = 0.2504, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.914740
Epoch 205
Validation binary_cross_entropy = 0.904121
Epoch 206
Validation binary_cross_entropy = 0.929316
Epoch 207
Validation binary_cross_entropy = 0.930038
Epoch 208
Validation binary_cross_entropy = 0.950133
Epoch 209
Loss = 1.3467e-02, PNorm = 71.1897, GNorm = 1.1555, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.982831
Epoch 210
Validation binary_cross_entropy = 1.011397
Epoch 211
Validation binary_cross_entropy = 1.004787
Epoch 212
Validation binary_cross_entropy = 0.962891
Epoch 213
Validation binary_cross_entropy = 0.933754
Epoch 214
Loss = 1.2931e-03, PNorm = 71.2391, GNorm = 0.1583, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.914825
Epoch 215
Validation binary_cross_entropy = 0.924331
Epoch 216
Validation binary_cross_entropy = 0.951738
Epoch 217
Validation binary_cross_entropy = 0.997833
Epoch 218
Validation binary_cross_entropy = 1.031556
Epoch 219
Loss = 3.4584e-03, PNorm = 71.2818, GNorm = 0.1291, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.046424
Epoch 220
Validation binary_cross_entropy = 1.043053
Epoch 221
Validation binary_cross_entropy = 1.030223
Epoch 222
Validation binary_cross_entropy = 0.973356
Epoch 223
Validation binary_cross_entropy = 0.929124
Epoch 224
Loss = 4.0961e-02, PNorm = 71.3261, GNorm = 4.1070, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.930986
Epoch 225
Validation binary_cross_entropy = 0.961635
Epoch 226
Validation binary_cross_entropy = 0.987017
Epoch 227
Validation binary_cross_entropy = 0.976477
Epoch 228
Validation binary_cross_entropy = 0.948555
Epoch 229
Loss = 8.4979e-04, PNorm = 71.3649, GNorm = 0.0730, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.929708
Epoch 230
Validation binary_cross_entropy = 0.930890
Epoch 231
Validation binary_cross_entropy = 0.956150
Epoch 232
Validation binary_cross_entropy = 0.981687
Epoch 233
Validation binary_cross_entropy = 0.996869
Epoch 234
Loss = 3.9810e-03, PNorm = 71.3948, GNorm = 0.2686, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.994890
Epoch 235
Validation binary_cross_entropy = 0.988803
Epoch 236
Validation binary_cross_entropy = 0.966522
Epoch 237
Validation binary_cross_entropy = 0.937367
Epoch 238
Validation binary_cross_entropy = 0.913917
Epoch 239
Loss = 1.3319e-02, PNorm = 71.4188, GNorm = 1.5949, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.913113
Epoch 240
Validation binary_cross_entropy = 0.932976
Epoch 241
Validation binary_cross_entropy = 0.958438
Epoch 242
Validation binary_cross_entropy = 0.991285
Epoch 243
Validation binary_cross_entropy = 1.026442
Epoch 244
Loss = 3.1586e-02, PNorm = 71.4491, GNorm = 3.5064, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.098454
Epoch 245
Validation binary_cross_entropy = 1.128178
Epoch 246
Validation binary_cross_entropy = 0.989843
Epoch 247
Validation binary_cross_entropy = 0.895904
Epoch 248
Validation binary_cross_entropy = 0.865441
Epoch 249
Loss = 1.6653e-02, PNorm = 71.5095, GNorm = 0.4881, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.862270
Epoch 250
Validation binary_cross_entropy = 0.852030
Epoch 251
Validation binary_cross_entropy = 0.858994
Epoch 252
Validation binary_cross_entropy = 0.889973
Epoch 253
Validation binary_cross_entropy = 0.917349
Epoch 254
Loss = 4.0634e-03, PNorm = 71.5846, GNorm = 0.3082, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.938122
Epoch 255
Validation binary_cross_entropy = 0.958151
Epoch 256
Validation binary_cross_entropy = 0.958098
Epoch 257
Validation binary_cross_entropy = 0.958024
Epoch 258
Validation binary_cross_entropy = 0.956878
Epoch 259
Loss = 2.7113e-03, PNorm = 71.6452, GNorm = 0.1732, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.960462
Epoch 260
Validation binary_cross_entropy = 0.963648
Epoch 261
Validation binary_cross_entropy = 0.986956
Epoch 262
Validation binary_cross_entropy = 1.044626
Epoch 263
Validation binary_cross_entropy = 1.098915
Epoch 264
Loss = 4.6537e-03, PNorm = 71.6899, GNorm = 0.1374, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.117198
Epoch 265
Validation binary_cross_entropy = 1.114417
Epoch 266
Validation binary_cross_entropy = 1.071696
Epoch 267
Validation binary_cross_entropy = 1.025902
Epoch 268
Validation binary_cross_entropy = 0.994314
Epoch 269
Loss = 1.1199e-03, PNorm = 71.7128, GNorm = 0.0120, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.973974
Epoch 270
Validation binary_cross_entropy = 0.961174
Epoch 271
Validation binary_cross_entropy = 0.954244
Epoch 272
Validation binary_cross_entropy = 0.962028
Epoch 273
Validation binary_cross_entropy = 0.972560
Epoch 274
Loss = 1.9292e-03, PNorm = 71.7331, GNorm = 0.2865, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.985423
Epoch 275
Validation binary_cross_entropy = 0.956637
Epoch 276
Validation binary_cross_entropy = 0.947598
Epoch 277
Validation binary_cross_entropy = 0.947035
Epoch 278
Validation binary_cross_entropy = 0.943675
Epoch 279
Loss = 3.6030e-02, PNorm = 71.7385, GNorm = 2.1816, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.945484
Epoch 280
Validation binary_cross_entropy = 0.949787
Epoch 281
Validation binary_cross_entropy = 0.958404
Epoch 282
Validation binary_cross_entropy = 0.971448
Epoch 283
Validation binary_cross_entropy = 0.998658
Epoch 284
Loss = 2.3778e-03, PNorm = 71.7805, GNorm = 0.1488, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.011582
Epoch 285
Validation binary_cross_entropy = 1.001139
Epoch 286
Validation binary_cross_entropy = 0.969494
Epoch 287
Validation binary_cross_entropy = 0.947139
Epoch 288
Validation binary_cross_entropy = 0.931817
Epoch 289
Loss = 1.1863e-02, PNorm = 71.7820, GNorm = 0.0208, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.948570
Epoch 290
Validation binary_cross_entropy = 0.973048
Epoch 291
Validation binary_cross_entropy = 1.001464
Epoch 292
Validation binary_cross_entropy = 1.058557
Epoch 293
Validation binary_cross_entropy = 1.085123
Epoch 294
Loss = 1.5240e-02, PNorm = 71.8055, GNorm = 1.0922, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.093443
Epoch 295
Validation binary_cross_entropy = 1.088775
Epoch 296
Validation binary_cross_entropy = 1.010112
Epoch 297
Validation binary_cross_entropy = 0.936398
Epoch 298
Validation binary_cross_entropy = 0.929082
Epoch 299
Loss = 1.1807e-01, PNorm = 71.9001, GNorm = 3.7458, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.965870
Model 0 best validation binary_cross_entropy = 0.336724 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.178040
Ensemble test binary_cross_entropy = 0.178040
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.400309
Epoch 1
Validation binary_cross_entropy = 1.047164
Epoch 2
Validation binary_cross_entropy = 0.892875
Epoch 3
Validation binary_cross_entropy = 0.380983
Epoch 4
Loss = 6.9217e-01, PNorm = 68.1549, GNorm = 11.5891, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.691377
Epoch 5
Validation binary_cross_entropy = 0.541637
Epoch 6
Validation binary_cross_entropy = 0.551926
Epoch 7
Validation binary_cross_entropy = 1.102741
Epoch 8
Validation binary_cross_entropy = 0.523897
Epoch 9
Loss = 5.9469e-01, PNorm = 68.3182, GNorm = 8.9341, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.524609
Epoch 10
Validation binary_cross_entropy = 0.889081
Epoch 11
Validation binary_cross_entropy = 0.527589
Epoch 12
Validation binary_cross_entropy = 0.545484
Epoch 13
Validation binary_cross_entropy = 0.622889
Epoch 14
Loss = 2.0483e-01, PNorm = 68.5015, GNorm = 2.5054, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.657858
Epoch 15
Validation binary_cross_entropy = 0.628861
Epoch 16
Validation binary_cross_entropy = 0.648937
Epoch 17
Validation binary_cross_entropy = 0.572605
Epoch 18
Validation binary_cross_entropy = 0.595426
Epoch 19
Loss = 1.6687e-01, PNorm = 68.6328, GNorm = 2.2285, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.538064
Epoch 20
Validation binary_cross_entropy = 0.630481
Epoch 21
Validation binary_cross_entropy = 0.542537
Epoch 22
Validation binary_cross_entropy = 0.610819
Epoch 23
Validation binary_cross_entropy = 0.619439
Epoch 24
Loss = 2.5068e-01, PNorm = 68.7216, GNorm = 6.3109, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.661052
Epoch 25
Validation binary_cross_entropy = 0.657124
Epoch 26
Validation binary_cross_entropy = 0.640759
Epoch 27
Validation binary_cross_entropy = 0.572862
Epoch 28
Validation binary_cross_entropy = 0.584066
Epoch 29
Loss = 1.8350e-01, PNorm = 68.7952, GNorm = 2.6046, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.646444
Epoch 30
Validation binary_cross_entropy = 0.705146
Epoch 31
Validation binary_cross_entropy = 0.588212
Epoch 32
Validation binary_cross_entropy = 0.846045
Epoch 33
Validation binary_cross_entropy = 0.666417
Epoch 34
Loss = 7.6006e-02, PNorm = 68.8668, GNorm = 3.6059, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.670797
Epoch 35
Validation binary_cross_entropy = 0.628418
Epoch 36
Validation binary_cross_entropy = 0.607732
Epoch 37
Validation binary_cross_entropy = 0.617755
Epoch 38
Validation binary_cross_entropy = 0.627037
Epoch 39
Loss = 8.5916e-02, PNorm = 68.9421, GNorm = 3.8562, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.659089
Epoch 40
Validation binary_cross_entropy = 0.546179
Epoch 41
Validation binary_cross_entropy = 0.703210
Epoch 42
Validation binary_cross_entropy = 0.616562
Epoch 43
Validation binary_cross_entropy = 0.719770
Epoch 44
Loss = 9.6939e-02, PNorm = 69.0270, GNorm = 3.9288, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.679419
Epoch 45
Validation binary_cross_entropy = 0.663601
Epoch 46
Validation binary_cross_entropy = 0.663765
Epoch 47
Validation binary_cross_entropy = 0.727852
Epoch 48
Validation binary_cross_entropy = 0.710416
Epoch 49
Loss = 8.1523e-02, PNorm = 69.1247, GNorm = 1.4924, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.628084
Epoch 50
Validation binary_cross_entropy = 0.651376
Epoch 51
Validation binary_cross_entropy = 0.673784
Epoch 52
Validation binary_cross_entropy = 0.753010
Epoch 53
Validation binary_cross_entropy = 0.725462
Epoch 54
Loss = 2.1515e-02, PNorm = 69.2122, GNorm = 0.5374, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.714487
Epoch 55
Validation binary_cross_entropy = 0.745155
Epoch 56
Validation binary_cross_entropy = 0.728306
Epoch 57
Validation binary_cross_entropy = 0.747531
Epoch 58
Validation binary_cross_entropy = 0.690633
Epoch 59
Loss = 1.0146e-01, PNorm = 69.2853, GNorm = 2.7111, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.647210
Epoch 60
Validation binary_cross_entropy = 0.674030
Epoch 61
Validation binary_cross_entropy = 0.663955
Epoch 62
Validation binary_cross_entropy = 0.639314
Epoch 63
Validation binary_cross_entropy = 0.644051
Epoch 64
Loss = 3.3670e-02, PNorm = 69.3518, GNorm = 0.2262, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.677193
Epoch 65
Validation binary_cross_entropy = 0.696773
Epoch 66
Validation binary_cross_entropy = 0.695306
Epoch 67
Validation binary_cross_entropy = 0.696418
Epoch 68
Validation binary_cross_entropy = 0.693465
Epoch 69
Loss = 6.4404e-03, PNorm = 69.4235, GNorm = 0.1890, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.690362
Epoch 70
Validation binary_cross_entropy = 0.718571
Epoch 71
Validation binary_cross_entropy = 0.754955
Epoch 72
Validation binary_cross_entropy = 0.724359
Epoch 73
Validation binary_cross_entropy = 0.692080
Epoch 74
Loss = 7.8901e-03, PNorm = 69.4720, GNorm = 0.0921, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.685064
Epoch 75
Validation binary_cross_entropy = 0.681010
Epoch 76
Validation binary_cross_entropy = 0.763366
Epoch 77
Validation binary_cross_entropy = 0.790637
Epoch 78
Validation binary_cross_entropy = 0.668028
Epoch 79
Loss = 1.1660e-01, PNorm = 69.5570, GNorm = 7.0444, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.635094
Epoch 80
Validation binary_cross_entropy = 0.655212
Epoch 81
Validation binary_cross_entropy = 0.715628
Epoch 82
Validation binary_cross_entropy = 0.775611
Epoch 83
Validation binary_cross_entropy = 0.689949
Epoch 84
Loss = 4.4714e-02, PNorm = 69.6903, GNorm = 2.1376, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.629113
Epoch 85
Validation binary_cross_entropy = 0.619033
Epoch 86
Validation binary_cross_entropy = 0.631842
Epoch 87
Validation binary_cross_entropy = 0.642830
Epoch 88
Validation binary_cross_entropy = 0.660601
Epoch 89
Loss = 3.0125e-02, PNorm = 69.8467, GNorm = 0.6224, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.651854
Epoch 90
Validation binary_cross_entropy = 0.648365
Epoch 91
Validation binary_cross_entropy = 0.660008
Epoch 92
Validation binary_cross_entropy = 0.683738
Epoch 93
Validation binary_cross_entropy = 0.702809
Epoch 94
Loss = 7.7286e-03, PNorm = 69.9373, GNorm = 0.2302, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.707845
Epoch 95
Validation binary_cross_entropy = 0.714983
Epoch 96
Validation binary_cross_entropy = 0.733971
Epoch 97
Validation binary_cross_entropy = 0.762038
Epoch 98
Validation binary_cross_entropy = 0.767364
Epoch 99
Loss = 1.8401e-02, PNorm = 69.9832, GNorm = 0.9517, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.755885
Epoch 100
Validation binary_cross_entropy = 0.765103
Epoch 101
Validation binary_cross_entropy = 0.773322
Epoch 102
Validation binary_cross_entropy = 0.776494
Epoch 103
Validation binary_cross_entropy = 0.771549
Epoch 104
Loss = 2.6037e-02, PNorm = 70.0154, GNorm = 1.8154, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.734432
Epoch 105
Validation binary_cross_entropy = 0.702969
Epoch 106
Validation binary_cross_entropy = 0.704079
Epoch 107
Validation binary_cross_entropy = 0.733241
Epoch 108
Validation binary_cross_entropy = 0.766095
Epoch 109
Loss = 1.8818e-02, PNorm = 70.0493, GNorm = 0.8462, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.792056
Epoch 110
Validation binary_cross_entropy = 0.766116
Epoch 111
Validation binary_cross_entropy = 0.729702
Epoch 112
Validation binary_cross_entropy = 0.712440
Epoch 113
Validation binary_cross_entropy = 0.699662
Epoch 114
Loss = 1.1155e-02, PNorm = 70.0779, GNorm = 0.7003, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.695525
Epoch 115
Validation binary_cross_entropy = 0.709430
Epoch 116
Validation binary_cross_entropy = 0.768900
Epoch 117
Validation binary_cross_entropy = 0.802803
Epoch 118
Validation binary_cross_entropy = 0.794058
Epoch 119
Loss = 1.3992e-02, PNorm = 70.1095, GNorm = 0.9254, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.769094
Epoch 120
Validation binary_cross_entropy = 0.767015
Epoch 121
Validation binary_cross_entropy = 0.795071
Epoch 122
Validation binary_cross_entropy = 0.769969
Epoch 123
Validation binary_cross_entropy = 0.737916
Epoch 124
Loss = 3.1136e-03, PNorm = 70.1433, GNorm = 0.0628, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.741566
Epoch 125
Validation binary_cross_entropy = 0.760372
Epoch 126
Validation binary_cross_entropy = 0.794651
Epoch 127
Validation binary_cross_entropy = 0.802201
Epoch 128
Validation binary_cross_entropy = 0.768828
Epoch 129
Loss = 2.3512e-03, PNorm = 70.1736, GNorm = 0.0863, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.737137
Epoch 130
Validation binary_cross_entropy = 0.748563
Epoch 131
Validation binary_cross_entropy = 0.783475
Epoch 132
Validation binary_cross_entropy = 0.823807
Epoch 133
Validation binary_cross_entropy = 0.788569
Epoch 134
Loss = 1.3603e-02, PNorm = 70.2086, GNorm = 1.6128, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.760276
Epoch 135
Validation binary_cross_entropy = 0.748137
Epoch 136
Validation binary_cross_entropy = 0.745762
Epoch 137
Validation binary_cross_entropy = 0.750561
Epoch 138
Validation binary_cross_entropy = 0.775128
Epoch 139
Loss = 1.3555e-02, PNorm = 70.2420, GNorm = 0.1525, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.794437
Epoch 140
Validation binary_cross_entropy = 0.792839
Epoch 141
Validation binary_cross_entropy = 0.797434
Epoch 142
Validation binary_cross_entropy = 0.803385
Epoch 143
Validation binary_cross_entropy = 0.797920
Epoch 144
Loss = 9.1906e-03, PNorm = 70.2753, GNorm = 0.3392, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.781196
Epoch 145
Validation binary_cross_entropy = 0.785196
Epoch 146
Validation binary_cross_entropy = 0.802365
Epoch 147
Validation binary_cross_entropy = 0.820313
Epoch 148
Validation binary_cross_entropy = 0.888293
Epoch 149
Loss = 8.0971e-03, PNorm = 70.3128, GNorm = 0.7738, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.918890
Epoch 150
Validation binary_cross_entropy = 0.851807
Epoch 151
Validation binary_cross_entropy = 0.785176
Epoch 152
Validation binary_cross_entropy = 0.749759
Epoch 153
Validation binary_cross_entropy = 0.748241
Epoch 154
Loss = 2.2177e-02, PNorm = 70.3415, GNorm = 0.8882, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.781478
Epoch 155
Validation binary_cross_entropy = 0.838311
Epoch 156
Validation binary_cross_entropy = 0.889759
Epoch 157
Validation binary_cross_entropy = 0.901900
Epoch 158
Validation binary_cross_entropy = 0.882130
Epoch 159
Loss = 8.5587e-03, PNorm = 70.3892, GNorm = 0.3812, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.831758
Epoch 160
Validation binary_cross_entropy = 0.798958
Epoch 161
Validation binary_cross_entropy = 0.781659
Epoch 162
Validation binary_cross_entropy = 0.796500
Epoch 163
Validation binary_cross_entropy = 0.855472
Epoch 164
Loss = 1.7404e-02, PNorm = 70.4210, GNorm = 0.4835, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.874906
Epoch 165
Validation binary_cross_entropy = 0.885582
Epoch 166
Validation binary_cross_entropy = 0.878557
Epoch 167
Validation binary_cross_entropy = 0.906853
Epoch 168
Validation binary_cross_entropy = 0.911539
Epoch 169
Loss = 7.4734e-03, PNorm = 70.4531, GNorm = 0.4119, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.876040
Epoch 170
Validation binary_cross_entropy = 0.840106
Epoch 171
Validation binary_cross_entropy = 0.814765
Epoch 172
Validation binary_cross_entropy = 0.839463
Epoch 173
Validation binary_cross_entropy = 0.911102
Epoch 174
Loss = 1.0202e-02, PNorm = 70.4911, GNorm = 0.1940, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.931042
Epoch 175
Validation binary_cross_entropy = 0.881440
Epoch 176
Validation binary_cross_entropy = 0.826792
Epoch 177
Validation binary_cross_entropy = 0.801652
Epoch 178
Validation binary_cross_entropy = 0.803856
Epoch 179
Loss = 8.9274e-03, PNorm = 70.5335, GNorm = 1.2857, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.809482
Epoch 180
Validation binary_cross_entropy = 0.829208
Epoch 181
Validation binary_cross_entropy = 0.859505
Epoch 182
Validation binary_cross_entropy = 0.875876
Epoch 183
Validation binary_cross_entropy = 0.878221
Epoch 184
Loss = 3.4929e-02, PNorm = 70.5746, GNorm = 0.0854, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.840607
Epoch 185
Validation binary_cross_entropy = 0.810788
Epoch 186
Validation binary_cross_entropy = 0.797313
Epoch 187
Validation binary_cross_entropy = 0.799574
Epoch 188
Validation binary_cross_entropy = 0.830362
Epoch 189
Loss = 6.7297e-03, PNorm = 70.6181, GNorm = 0.3139, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.872041
Epoch 190
Validation binary_cross_entropy = 0.889027
Epoch 191
Validation binary_cross_entropy = 0.870256
Epoch 192
Validation binary_cross_entropy = 0.833161
Epoch 193
Validation binary_cross_entropy = 0.807257
Epoch 194
Loss = 3.5064e-03, PNorm = 70.6523, GNorm = 0.1250, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.785759
Epoch 195
Validation binary_cross_entropy = 0.796522
Epoch 196
Validation binary_cross_entropy = 0.828234
Epoch 197
Validation binary_cross_entropy = 0.866513
Epoch 198
Validation binary_cross_entropy = 0.887323
Epoch 199
Loss = 4.6187e-03, PNorm = 70.6876, GNorm = 0.1714, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.894351
Epoch 200
Validation binary_cross_entropy = 0.872089
Epoch 201
Validation binary_cross_entropy = 0.821366
Epoch 202
Validation binary_cross_entropy = 0.789751
Epoch 203
Validation binary_cross_entropy = 0.781730
Epoch 204
Loss = 3.6470e-03, PNorm = 70.7221, GNorm = 0.7619, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.786079
Epoch 205
Validation binary_cross_entropy = 0.797443
Epoch 206
Validation binary_cross_entropy = 0.818428
Epoch 207
Validation binary_cross_entropy = 0.865087
Epoch 208
Validation binary_cross_entropy = 0.903299
Epoch 209
Loss = 3.0191e-03, PNorm = 70.7680, GNorm = 0.2285, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.917352
Epoch 210
Validation binary_cross_entropy = 0.931201
Epoch 211
Validation binary_cross_entropy = 0.909390
Epoch 212
Validation binary_cross_entropy = 0.880824
Epoch 213
Validation binary_cross_entropy = 0.859053
Epoch 214
Loss = 2.2735e-03, PNorm = 70.7992, GNorm = 0.1419, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.839969
Epoch 215
Validation binary_cross_entropy = 0.827110
Epoch 216
Validation binary_cross_entropy = 0.823222
Epoch 217
Validation binary_cross_entropy = 0.821432
Epoch 218
Validation binary_cross_entropy = 0.825504
Epoch 219
Loss = 2.6576e-03, PNorm = 70.8246, GNorm = 0.0215, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.840443
Epoch 220
Validation binary_cross_entropy = 0.883688
Epoch 221
Validation binary_cross_entropy = 0.922229
Epoch 222
Validation binary_cross_entropy = 0.914499
Epoch 223
Validation binary_cross_entropy = 0.844756
Epoch 224
Loss = 2.3785e-03, PNorm = 70.8577, GNorm = 0.2050, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.826497
Epoch 225
Validation binary_cross_entropy = 0.862729
Epoch 226
Validation binary_cross_entropy = 0.877026
Epoch 227
Validation binary_cross_entropy = 0.972166
Epoch 228
Validation binary_cross_entropy = 1.050783
Epoch 229
Loss = 2.4059e-02, PNorm = 70.9211, GNorm = 0.4734, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.972022
Epoch 230
Validation binary_cross_entropy = 0.912835
Epoch 231
Validation binary_cross_entropy = 0.899415
Epoch 232
Validation binary_cross_entropy = 0.928435
Epoch 233
Validation binary_cross_entropy = 0.992020
Epoch 234
Loss = 9.2356e-03, PNorm = 71.0030, GNorm = 0.2899, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.056204
Epoch 235
Validation binary_cross_entropy = 1.265883
Epoch 236
Validation binary_cross_entropy = 0.869958
Epoch 237
Validation binary_cross_entropy = 0.819100
Epoch 238
Validation binary_cross_entropy = 0.796408
Epoch 239
Loss = 4.4148e-02, PNorm = 71.1862, GNorm = 2.6433, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.780362
Epoch 240
Validation binary_cross_entropy = 0.885807
Epoch 241
Validation binary_cross_entropy = 0.957119
Epoch 242
Validation binary_cross_entropy = 0.913328
Epoch 243
Validation binary_cross_entropy = 0.896715
Epoch 244
Loss = 1.3359e-02, PNorm = 71.3910, GNorm = 1.6093, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.945258
Epoch 245
Validation binary_cross_entropy = 1.028318
Epoch 246
Validation binary_cross_entropy = 1.021483
Epoch 247
Validation binary_cross_entropy = 1.025902
Epoch 248
Validation binary_cross_entropy = 1.078823
Epoch 249
Loss = 1.9446e-02, PNorm = 71.5316, GNorm = 1.2805, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.116651
Epoch 250
Validation binary_cross_entropy = 1.145614
Epoch 251
Validation binary_cross_entropy = 1.132237
Epoch 252
Validation binary_cross_entropy = 1.072795
Epoch 253
Validation binary_cross_entropy = 1.024164
Epoch 254
Loss = 1.8165e-03, PNorm = 71.6261, GNorm = 0.1100, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.992867
Epoch 255
Validation binary_cross_entropy = 0.974603
Epoch 256
Validation binary_cross_entropy = 0.968574
Epoch 257
Validation binary_cross_entropy = 0.963353
Epoch 258
Validation binary_cross_entropy = 0.982618
Epoch 259
Loss = 1.4741e-02, PNorm = 71.6741, GNorm = 1.3529, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.065933
Epoch 260
Validation binary_cross_entropy = 1.008772
Epoch 261
Validation binary_cross_entropy = 0.920867
Epoch 262
Validation binary_cross_entropy = 0.885931
Epoch 263
Validation binary_cross_entropy = 0.939848
Epoch 264
Loss = 2.9991e-03, PNorm = 71.7807, GNorm = 0.1067, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.982780
Epoch 265
Validation binary_cross_entropy = 1.004307
Epoch 266
Validation binary_cross_entropy = 1.014911
Epoch 267
Validation binary_cross_entropy = 1.000678
Epoch 268
Validation binary_cross_entropy = 0.977874
Epoch 269
Loss = 1.1425e-02, PNorm = 71.8571, GNorm = 1.2655, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.983361
Epoch 270
Validation binary_cross_entropy = 0.986209
Epoch 271
Validation binary_cross_entropy = 0.974491
Epoch 272
Validation binary_cross_entropy = 0.967687
Epoch 273
Validation binary_cross_entropy = 0.950978
Epoch 274
Loss = 9.7848e-03, PNorm = 71.9012, GNorm = 0.9853, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.952826
Epoch 275
Validation binary_cross_entropy = 0.966543
Epoch 276
Validation binary_cross_entropy = 0.990934
Epoch 277
Validation binary_cross_entropy = 1.028310
Epoch 278
Validation binary_cross_entropy = 1.068706
Epoch 279
Loss = 5.8262e-03, PNorm = 71.9411, GNorm = 0.8920, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.090750
Epoch 280
Validation binary_cross_entropy = 1.054595
Epoch 281
Validation binary_cross_entropy = 0.993268
Epoch 282
Validation binary_cross_entropy = 1.018597
Epoch 283
Validation binary_cross_entropy = 1.040014
Epoch 284
Loss = 1.4193e-02, PNorm = 71.9689, GNorm = 2.5764, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.016839
Epoch 285
Validation binary_cross_entropy = 0.912747
Epoch 286
Validation binary_cross_entropy = 0.878821
Epoch 287
Validation binary_cross_entropy = 0.885585
Epoch 288
Validation binary_cross_entropy = 0.901102
Epoch 289
Loss = 1.9845e-01, PNorm = 72.0433, GNorm = 6.9995, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.055714
Epoch 290
Validation binary_cross_entropy = 1.303049
Epoch 291
Validation binary_cross_entropy = 1.184270
Epoch 292
Validation binary_cross_entropy = 1.011132
Epoch 293
Validation binary_cross_entropy = 0.927579
Epoch 294
Loss = 3.5767e-02, PNorm = 72.1680, GNorm = 2.4847, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.935618
Epoch 295
Validation binary_cross_entropy = 0.983857
Epoch 296
Validation binary_cross_entropy = 1.086573
Epoch 297
Validation binary_cross_entropy = 1.152367
Epoch 298
Validation binary_cross_entropy = 1.092952
Epoch 299
Loss = 1.7545e-03, PNorm = 72.3198, GNorm = 0.0391, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.025480
Model 0 best validation binary_cross_entropy = 0.380983 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.188028
Ensemble test binary_cross_entropy = 0.188028
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.490596
Epoch 1
Validation binary_cross_entropy = 0.583997
Epoch 2
Validation binary_cross_entropy = 0.494077
Epoch 3
Validation binary_cross_entropy = 0.663706
Epoch 4
Loss = 4.7028e-01, PNorm = 68.1506, GNorm = 6.0214, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.542691
Epoch 5
Validation binary_cross_entropy = 0.631783
Epoch 6
Validation binary_cross_entropy = 0.439071
Epoch 7
Validation binary_cross_entropy = 0.460566
Epoch 8
Validation binary_cross_entropy = 0.600053
Epoch 9
Loss = 4.3658e-01, PNorm = 68.3098, GNorm = 6.7142, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.671989
Epoch 10
Validation binary_cross_entropy = 0.824047
Epoch 11
Validation binary_cross_entropy = 0.724172
Epoch 12
Validation binary_cross_entropy = 0.618602
Epoch 13
Validation binary_cross_entropy = 0.541114
Epoch 14
Loss = 2.3234e-01, PNorm = 68.4872, GNorm = 4.7040, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.615061
Epoch 15
Validation binary_cross_entropy = 0.553569
Epoch 16
Validation binary_cross_entropy = 0.600921
Epoch 17
Validation binary_cross_entropy = 0.810493
Epoch 18
Validation binary_cross_entropy = 0.554671
Epoch 19
Loss = 2.0791e-01, PNorm = 68.6185, GNorm = 6.4373, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.578277
Epoch 20
Validation binary_cross_entropy = 0.699711
Epoch 21
Validation binary_cross_entropy = 0.572485
Epoch 22
Validation binary_cross_entropy = 0.562548
Epoch 23
Validation binary_cross_entropy = 0.615142
Epoch 24
Loss = 1.5207e-01, PNorm = 68.7293, GNorm = 3.3987, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.533410
Epoch 25
Validation binary_cross_entropy = 0.553637
Epoch 26
Validation binary_cross_entropy = 0.596231
Epoch 27
Validation binary_cross_entropy = 0.641042
Epoch 28
Validation binary_cross_entropy = 0.602776
Epoch 29
Loss = 1.0961e-01, PNorm = 68.8280, GNorm = 4.7955, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.601591
Epoch 30
Validation binary_cross_entropy = 0.713210
Epoch 31
Validation binary_cross_entropy = 0.700361
Epoch 32
Validation binary_cross_entropy = 0.673869
Epoch 33
Validation binary_cross_entropy = 0.627179
Epoch 34
Loss = 1.1329e-01, PNorm = 68.9226, GNorm = 1.3617, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.557966
Epoch 35
Validation binary_cross_entropy = 0.638898
Epoch 36
Validation binary_cross_entropy = 0.666102
Epoch 37
Validation binary_cross_entropy = 0.999279
Epoch 38
Validation binary_cross_entropy = 1.115674
Epoch 39
Loss = 2.6995e-01, PNorm = 69.0199, GNorm = 11.1134, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.843645
Epoch 40
Validation binary_cross_entropy = 0.884542
Epoch 41
Validation binary_cross_entropy = 0.987426
Epoch 42
Validation binary_cross_entropy = 1.059272
Epoch 43
Validation binary_cross_entropy = 0.688617
Epoch 44
Loss = 2.1744e-01, PNorm = 69.1743, GNorm = 8.8978, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.820101
Epoch 45
Validation binary_cross_entropy = 0.645668
Epoch 46
Validation binary_cross_entropy = 0.785438
Epoch 47
Validation binary_cross_entropy = 0.731877
Epoch 48
Validation binary_cross_entropy = 0.649129
Epoch 49
Loss = 1.1879e-01, PNorm = 69.3393, GNorm = 3.5111, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.669835
Epoch 50
Validation binary_cross_entropy = 0.641269
Epoch 51
Validation binary_cross_entropy = 0.618808
Epoch 52
Validation binary_cross_entropy = 0.622670
Epoch 53
Validation binary_cross_entropy = 0.637698
Epoch 54
Loss = 4.6416e-02, PNorm = 69.4544, GNorm = 3.4784, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.639399
Epoch 55
Validation binary_cross_entropy = 0.646099
Epoch 56
Validation binary_cross_entropy = 0.714990
Epoch 57
Validation binary_cross_entropy = 0.669613
Epoch 58
Validation binary_cross_entropy = 0.625331
Epoch 59
Loss = 1.6519e-02, PNorm = 69.5606, GNorm = 0.2513, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.626592
Epoch 60
Validation binary_cross_entropy = 0.642558
Epoch 61
Validation binary_cross_entropy = 0.657951
Epoch 62
Validation binary_cross_entropy = 0.688915
Epoch 63
Validation binary_cross_entropy = 0.703229
Epoch 64
Loss = 3.3946e-02, PNorm = 69.6295, GNorm = 0.5664, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.683105
Epoch 65
Validation binary_cross_entropy = 0.669236
Epoch 66
Validation binary_cross_entropy = 0.672376
Epoch 67
Validation binary_cross_entropy = 0.679283
Epoch 68
Validation binary_cross_entropy = 0.661089
Epoch 69
Loss = 4.4056e-02, PNorm = 69.6714, GNorm = 2.2408, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.646662
Epoch 70
Validation binary_cross_entropy = 0.654780
Epoch 71
Validation binary_cross_entropy = 0.683846
Epoch 72
Validation binary_cross_entropy = 0.683376
Epoch 73
Validation binary_cross_entropy = 0.689255
Epoch 74
Loss = 1.1423e-02, PNorm = 69.7000, GNorm = 0.5922, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.699322
Epoch 75
Validation binary_cross_entropy = 0.703267
Epoch 76
Validation binary_cross_entropy = 0.706291
Epoch 77
Validation binary_cross_entropy = 0.714263
Epoch 78
Validation binary_cross_entropy = 0.727593
Epoch 79
Loss = 4.5357e-03, PNorm = 69.7187, GNorm = 0.1540, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.754660
Epoch 80
Validation binary_cross_entropy = 0.806683
Epoch 81
Validation binary_cross_entropy = 0.820830
Epoch 82
Validation binary_cross_entropy = 0.774080
Epoch 83
Validation binary_cross_entropy = 0.745241
Epoch 84
Loss = 7.9170e-03, PNorm = 69.7438, GNorm = 0.3953, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.723434
Epoch 85
Validation binary_cross_entropy = 0.726778
Epoch 86
Validation binary_cross_entropy = 0.704579
Epoch 87
Validation binary_cross_entropy = 0.690484
Epoch 88
Validation binary_cross_entropy = 0.679854
Epoch 89
Loss = 4.6798e-02, PNorm = 69.7716, GNorm = 3.1186, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.691501
Epoch 90
Validation binary_cross_entropy = 0.753062
Epoch 91
Validation binary_cross_entropy = 0.749066
Epoch 92
Validation binary_cross_entropy = 0.708196
Epoch 93
Validation binary_cross_entropy = 0.686938
Epoch 94
Loss = 4.3836e-03, PNorm = 69.8008, GNorm = 0.2223, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.680573
Epoch 95
Validation binary_cross_entropy = 0.690086
Epoch 96
Validation binary_cross_entropy = 0.713790
Epoch 97
Validation binary_cross_entropy = 0.769059
Epoch 98
Validation binary_cross_entropy = 0.813811
Epoch 99
Loss = 2.3749e-02, PNorm = 69.8313, GNorm = 2.2648, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.804041
Epoch 100
Validation binary_cross_entropy = 0.750969
Epoch 101
Validation binary_cross_entropy = 0.711684
Epoch 102
Validation binary_cross_entropy = 0.692187
Epoch 103
Validation binary_cross_entropy = 0.693178
Epoch 104
Loss = 2.4338e-02, PNorm = 69.8592, GNorm = 0.8845, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.730751
Epoch 105
Validation binary_cross_entropy = 0.784287
Epoch 106
Validation binary_cross_entropy = 0.769794
Epoch 107
Validation binary_cross_entropy = 0.732748
Epoch 108
Validation binary_cross_entropy = 0.722507
Epoch 109
Loss = 6.6647e-03, PNorm = 69.8905, GNorm = 0.2092, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.718200
Epoch 110
Validation binary_cross_entropy = 0.714577
Epoch 111
Validation binary_cross_entropy = 0.725289
Epoch 112
Validation binary_cross_entropy = 0.745388
Epoch 113
Validation binary_cross_entropy = 0.748198
Epoch 114
Loss = 5.3216e-03, PNorm = 69.9169, GNorm = 0.0661, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.741641
Epoch 115
Validation binary_cross_entropy = 0.724246
Epoch 116
Validation binary_cross_entropy = 0.722268
Epoch 117
Validation binary_cross_entropy = 0.734915
Epoch 118
Validation binary_cross_entropy = 0.751504
Epoch 119
Loss = 2.1569e-02, PNorm = 69.9410, GNorm = 0.1765, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.793469
Epoch 120
Validation binary_cross_entropy = 0.835085
Epoch 121
Validation binary_cross_entropy = 0.836151
Epoch 122
Validation binary_cross_entropy = 0.800094
Epoch 123
Validation binary_cross_entropy = 0.774346
Epoch 124
Loss = 1.5909e-02, PNorm = 69.9661, GNorm = 0.0865, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.782845
Epoch 125
Validation binary_cross_entropy = 0.788625
Epoch 126
Validation binary_cross_entropy = 0.777034
Epoch 127
Validation binary_cross_entropy = 0.763726
Epoch 128
Validation binary_cross_entropy = 0.744681
Epoch 129
Loss = 8.6826e-02, PNorm = 69.9907, GNorm = 3.7754, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.758748
Epoch 130
Validation binary_cross_entropy = 0.849539
Epoch 131
Validation binary_cross_entropy = 0.941213
Epoch 132
Validation binary_cross_entropy = 0.907617
Epoch 133
Validation binary_cross_entropy = 0.830224
Epoch 134
Loss = 4.3459e-03, PNorm = 70.0286, GNorm = 0.0752, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.779481
Epoch 135
Validation binary_cross_entropy = 0.777679
Epoch 136
Validation binary_cross_entropy = 0.790700
Epoch 137
Validation binary_cross_entropy = 0.820010
Epoch 138
Validation binary_cross_entropy = 0.849246
Epoch 139
Loss = 1.2685e-02, PNorm = 70.0648, GNorm = 0.6848, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.849025
Epoch 140
Validation binary_cross_entropy = 0.819830
Epoch 141
Validation binary_cross_entropy = 0.791889
Epoch 142
Validation binary_cross_entropy = 0.806861
Epoch 143
Validation binary_cross_entropy = 0.819065
Epoch 144
Loss = 5.2990e-03, PNorm = 70.1030, GNorm = 0.4870, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.823022
Epoch 145
Validation binary_cross_entropy = 0.817044
Epoch 146
Validation binary_cross_entropy = 0.797553
Epoch 147
Validation binary_cross_entropy = 0.766221
Epoch 148
Validation binary_cross_entropy = 0.748086
Epoch 149
Loss = 1.3592e-02, PNorm = 70.1808, GNorm = 0.6531, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.755414
Epoch 150
Validation binary_cross_entropy = 0.777421
Epoch 151
Validation binary_cross_entropy = 0.808081
Epoch 152
Validation binary_cross_entropy = 0.810769
Epoch 153
Validation binary_cross_entropy = 0.808070
Epoch 154
Loss = 3.3013e-02, PNorm = 70.2402, GNorm = 3.5183, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.826872
Epoch 155
Validation binary_cross_entropy = 0.883753
Epoch 156
Validation binary_cross_entropy = 0.907696
Epoch 157
Validation binary_cross_entropy = 0.840485
Epoch 158
Validation binary_cross_entropy = 0.784079
Epoch 159
Loss = 1.6121e-02, PNorm = 70.2942, GNorm = 1.0048, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.777130
Epoch 160
Validation binary_cross_entropy = 0.784590
Epoch 161
Validation binary_cross_entropy = 0.864978
Epoch 162
Validation binary_cross_entropy = 0.841681
Epoch 163
Validation binary_cross_entropy = 0.815243
Epoch 164
Loss = 1.2726e-02, PNorm = 70.3593, GNorm = 0.0708, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.779593
Epoch 165
Validation binary_cross_entropy = 0.782492
Epoch 166
Validation binary_cross_entropy = 0.799169
Epoch 167
Validation binary_cross_entropy = 0.845865
Epoch 168
Validation binary_cross_entropy = 0.911153
Epoch 169
Loss = 1.0186e-02, PNorm = 70.4111, GNorm = 0.7490, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.948056
Epoch 170
Validation binary_cross_entropy = 0.845331
Epoch 171
Validation binary_cross_entropy = 0.776855
Epoch 172
Validation binary_cross_entropy = 0.742169
Epoch 173
Validation binary_cross_entropy = 0.731298
Epoch 174
Loss = 7.2303e-03, PNorm = 70.4665, GNorm = 0.9619, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.731503
Epoch 175
Validation binary_cross_entropy = 0.781837
Epoch 176
Validation binary_cross_entropy = 0.819619
Epoch 177
Validation binary_cross_entropy = 0.839902
Epoch 178
Validation binary_cross_entropy = 0.822491
Epoch 179
Loss = 2.2181e-02, PNorm = 70.5339, GNorm = 0.1467, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.765431
Epoch 180
Validation binary_cross_entropy = 0.749402
Epoch 181
Validation binary_cross_entropy = 0.766931
Epoch 182
Validation binary_cross_entropy = 0.786759
Epoch 183
Validation binary_cross_entropy = 0.757042
Epoch 184
Loss = 1.7431e-02, PNorm = 70.5964, GNorm = 1.6692, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.745351
Epoch 185
Validation binary_cross_entropy = 0.758141
Epoch 186
Validation binary_cross_entropy = 0.778286
Epoch 187
Validation binary_cross_entropy = 0.841157
Epoch 188
Validation binary_cross_entropy = 0.909631
Epoch 189
Loss = 3.2599e-02, PNorm = 70.6594, GNorm = 1.5690, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.907403
Epoch 190
Validation binary_cross_entropy = 0.910549
Epoch 191
Validation binary_cross_entropy = 0.917911
Epoch 192
Validation binary_cross_entropy = 0.884658
Epoch 193
Validation binary_cross_entropy = 0.872913
Epoch 194
Loss = 4.3756e-03, PNorm = 70.7046, GNorm = 0.1676, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.876588
Epoch 195
Validation binary_cross_entropy = 0.902090
Epoch 196
Validation binary_cross_entropy = 0.946839
Epoch 197
Validation binary_cross_entropy = 0.914895
Epoch 198
Validation binary_cross_entropy = 0.911600
Epoch 199
Loss = 7.2874e-03, PNorm = 70.7506, GNorm = 0.1373, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.949836
Epoch 200
Validation binary_cross_entropy = 0.975495
Epoch 201
Validation binary_cross_entropy = 0.985141
Epoch 202
Validation binary_cross_entropy = 0.946645
Epoch 203
Validation binary_cross_entropy = 0.915604
Epoch 204
Loss = 3.3529e-03, PNorm = 70.7838, GNorm = 0.3372, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.896144
Epoch 205
Validation binary_cross_entropy = 0.889616
Epoch 206
Validation binary_cross_entropy = 0.882061
Epoch 207
Validation binary_cross_entropy = 0.878273
Epoch 208
Validation binary_cross_entropy = 0.880014
Epoch 209
Loss = 6.6845e-02, PNorm = 70.8281, GNorm = 4.1904, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.977677
Epoch 210
Validation binary_cross_entropy = 1.090093
Epoch 211
Validation binary_cross_entropy = 0.966271
Epoch 212
Validation binary_cross_entropy = 0.877121
Epoch 213
Validation binary_cross_entropy = 0.864108
Epoch 214
Loss = 1.1699e-02, PNorm = 70.9087, GNorm = 2.0035, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.967297
Epoch 215
Validation binary_cross_entropy = 0.934127
Epoch 216
Validation binary_cross_entropy = 0.895190
Epoch 217
Validation binary_cross_entropy = 0.925944
Epoch 218
Validation binary_cross_entropy = 0.961575
Epoch 219
Loss = 2.4903e-02, PNorm = 70.9858, GNorm = 0.6577, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.908206
Epoch 220
Validation binary_cross_entropy = 0.907979
Epoch 221
Validation binary_cross_entropy = 0.882637
Epoch 222
Validation binary_cross_entropy = 0.858823
Epoch 223
Validation binary_cross_entropy = 0.860793
Epoch 224
Loss = 7.8665e-02, PNorm = 71.0865, GNorm = 3.6919, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.873118
Epoch 225
Validation binary_cross_entropy = 0.912822
Epoch 226
Validation binary_cross_entropy = 0.950629
Epoch 227
Validation binary_cross_entropy = 0.965359
Epoch 228
Validation binary_cross_entropy = 0.943818
Epoch 229
Loss = 1.2892e-03, PNorm = 71.1711, GNorm = 0.0342, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.923631
Epoch 230
Validation binary_cross_entropy = 0.912078
Epoch 231
Validation binary_cross_entropy = 0.918728
Epoch 232
Validation binary_cross_entropy = 0.927769
Epoch 233
Validation binary_cross_entropy = 0.980188
Epoch 234
Loss = 1.0366e-02, PNorm = 71.2673, GNorm = 0.6266, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.050354
Epoch 235
Validation binary_cross_entropy = 1.042346
Epoch 236
Validation binary_cross_entropy = 1.006935
Epoch 237
Validation binary_cross_entropy = 0.928299
Epoch 238
Validation binary_cross_entropy = 0.847345
Epoch 239
Loss = 7.0980e-03, PNorm = 71.3528, GNorm = 0.5174, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.821399
Epoch 240
Validation binary_cross_entropy = 0.820704
Epoch 241
Validation binary_cross_entropy = 0.837721
Epoch 242
Validation binary_cross_entropy = 0.863719
Epoch 243
Validation binary_cross_entropy = 0.908567
Epoch 244
Loss = 3.0692e-03, PNorm = 71.4262, GNorm = 0.1929, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.952165
Epoch 245
Validation binary_cross_entropy = 0.984381
Epoch 246
Validation binary_cross_entropy = 1.008188
Epoch 247
Validation binary_cross_entropy = 1.011981
Epoch 248
Validation binary_cross_entropy = 0.981639
Epoch 249
Loss = 5.2292e-03, PNorm = 71.4607, GNorm = 0.6375, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.939195
Epoch 250
Validation binary_cross_entropy = 0.898784
Epoch 251
Validation binary_cross_entropy = 0.879331
Epoch 252
Validation binary_cross_entropy = 0.873162
Epoch 253
Validation binary_cross_entropy = 0.875651
Epoch 254
Loss = 8.7136e-03, PNorm = 71.4833, GNorm = 1.1486, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.893291
Epoch 255
Validation binary_cross_entropy = 0.923120
Epoch 256
Validation binary_cross_entropy = 0.946957
Epoch 257
Validation binary_cross_entropy = 0.961834
Epoch 258
Validation binary_cross_entropy = 0.934276
Epoch 259
Loss = 3.2568e-03, PNorm = 71.5183, GNorm = 0.1832, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.912991
Epoch 260
Validation binary_cross_entropy = 0.891998
Epoch 261
Validation binary_cross_entropy = 0.852813
Epoch 262
Validation binary_cross_entropy = 0.838284
Epoch 263
Validation binary_cross_entropy = 0.839200
Epoch 264
Loss = 2.5067e-02, PNorm = 71.5396, GNorm = 0.1378, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.869293
Epoch 265
Validation binary_cross_entropy = 0.919102
Epoch 266
Validation binary_cross_entropy = 0.937628
Epoch 267
Validation binary_cross_entropy = 0.949736
Epoch 268
Validation binary_cross_entropy = 0.954806
Epoch 269
Loss = 1.1506e-02, PNorm = 71.5798, GNorm = 1.2111, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.967429
Epoch 270
Validation binary_cross_entropy = 0.984564
Epoch 271
Validation binary_cross_entropy = 0.987926
Epoch 272
Validation binary_cross_entropy = 0.985659
Epoch 273
Validation binary_cross_entropy = 0.970185
Epoch 274
Loss = 2.3099e-03, PNorm = 71.6351, GNorm = 0.0333, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.949793
Epoch 275
Validation binary_cross_entropy = 0.934661
Epoch 276
Validation binary_cross_entropy = 0.964450
Epoch 277
Validation binary_cross_entropy = 0.995379
Epoch 278
Validation binary_cross_entropy = 1.014341
Epoch 279
Loss = 3.8719e-03, PNorm = 71.6631, GNorm = 0.5150, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.034524
Epoch 280
Validation binary_cross_entropy = 1.016524
Epoch 281
Validation binary_cross_entropy = 0.995921
Epoch 282
Validation binary_cross_entropy = 1.005144
Epoch 283
Validation binary_cross_entropy = 1.011089
Epoch 284
Loss = 9.4535e-03, PNorm = 71.6925, GNorm = 1.8132, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.034714
Epoch 285
Validation binary_cross_entropy = 1.060729
Epoch 286
Validation binary_cross_entropy = 1.052269
Epoch 287
Validation binary_cross_entropy = 1.003242
Epoch 288
Validation binary_cross_entropy = 0.942831
Epoch 289
Loss = 1.0359e-03, PNorm = 71.7270, GNorm = 0.0910, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.909034
Epoch 290
Validation binary_cross_entropy = 0.901367
Epoch 291
Validation binary_cross_entropy = 0.914721
Epoch 292
Validation binary_cross_entropy = 0.932362
Epoch 293
Validation binary_cross_entropy = 0.951431
Epoch 294
Loss = 3.4698e-03, PNorm = 71.7874, GNorm = 0.2929, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.964815
Epoch 295
Validation binary_cross_entropy = 0.935942
Epoch 296
Validation binary_cross_entropy = 0.910678
Epoch 297
Validation binary_cross_entropy = 0.897051
Epoch 298
Validation binary_cross_entropy = 0.942901
Epoch 299
Loss = 1.2158e-03, PNorm = 71.8115, GNorm = 0.0890, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.989750
Model 0 best validation binary_cross_entropy = 0.439071 on epoch 6
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.311761
Ensemble test binary_cross_entropy = 0.311761
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.393258
Epoch 1
Validation binary_cross_entropy = 0.443853
Epoch 2
Validation binary_cross_entropy = 0.903837
Epoch 3
Validation binary_cross_entropy = 1.620155
Epoch 4
Loss = 9.3328e-01, PNorm = 68.1471, GNorm = 9.7188, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.479463
Epoch 5
Validation binary_cross_entropy = 0.569364
Epoch 6
Validation binary_cross_entropy = 1.733793
Epoch 7
Validation binary_cross_entropy = 0.972458
Epoch 8
Validation binary_cross_entropy = 0.636916
Epoch 9
Loss = 5.1699e-01, PNorm = 68.3132, GNorm = 11.4857, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.667883
Epoch 10
Validation binary_cross_entropy = 1.043519
Epoch 11
Validation binary_cross_entropy = 0.583630
Epoch 12
Validation binary_cross_entropy = 0.532048
Epoch 13
Validation binary_cross_entropy = 0.508775
Epoch 14
Loss = 1.6990e-01, PNorm = 68.4831, GNorm = 2.4036, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.817436
Epoch 15
Validation binary_cross_entropy = 0.644645
Epoch 16
Validation binary_cross_entropy = 0.527633
Epoch 17
Validation binary_cross_entropy = 0.561085
Epoch 18
Validation binary_cross_entropy = 0.524025
Epoch 19
Loss = 1.7868e-01, PNorm = 68.5961, GNorm = 3.3700, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.555616
Epoch 20
Validation binary_cross_entropy = 0.552498
Epoch 21
Validation binary_cross_entropy = 0.551712
Epoch 22
Validation binary_cross_entropy = 0.546988
Epoch 23
Validation binary_cross_entropy = 0.507113
Epoch 24
Loss = 1.5683e-01, PNorm = 68.6751, GNorm = 1.9747, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.485277
Epoch 25
Validation binary_cross_entropy = 0.486459
Epoch 26
Validation binary_cross_entropy = 0.516849
Epoch 27
Validation binary_cross_entropy = 0.510481
Epoch 28
Validation binary_cross_entropy = 0.510914
Epoch 29
Loss = 1.0266e-01, PNorm = 68.7352, GNorm = 1.7379, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.507335
Epoch 30
Validation binary_cross_entropy = 0.509647
Epoch 31
Validation binary_cross_entropy = 0.508745
Epoch 32
Validation binary_cross_entropy = 0.512958
Epoch 33
Validation binary_cross_entropy = 0.515412
Epoch 34
Loss = 9.2697e-02, PNorm = 68.7810, GNorm = 1.8707, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.512559
Epoch 35
Validation binary_cross_entropy = 0.525137
Epoch 36
Validation binary_cross_entropy = 0.578948
Epoch 37
Validation binary_cross_entropy = 0.585351
Epoch 38
Validation binary_cross_entropy = 0.538928
Epoch 39
Loss = 5.9722e-02, PNorm = 68.8477, GNorm = 0.6849, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.554792
Epoch 40
Validation binary_cross_entropy = 0.553475
Epoch 41
Validation binary_cross_entropy = 0.601119
Epoch 42
Validation binary_cross_entropy = 0.559910
Epoch 43
Validation binary_cross_entropy = 0.547629
Epoch 44
Loss = 4.0573e-02, PNorm = 68.9168, GNorm = 2.0302, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.557799
Epoch 45
Validation binary_cross_entropy = 0.566834
Epoch 46
Validation binary_cross_entropy = 0.612676
Epoch 47
Validation binary_cross_entropy = 0.609881
Epoch 48
Validation binary_cross_entropy = 0.543667
Epoch 49
Loss = 4.7056e-02, PNorm = 68.9902, GNorm = 0.4792, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.554472
Epoch 50
Validation binary_cross_entropy = 0.563767
Epoch 51
Validation binary_cross_entropy = 0.560406
Epoch 52
Validation binary_cross_entropy = 0.587152
Epoch 53
Validation binary_cross_entropy = 0.645502
Epoch 54
Loss = 4.5757e-02, PNorm = 69.0667, GNorm = 1.4690, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.631648
Epoch 55
Validation binary_cross_entropy = 0.637796
Epoch 56
Validation binary_cross_entropy = 0.636977
Epoch 57
Validation binary_cross_entropy = 0.598753
Epoch 58
Validation binary_cross_entropy = 0.593346
Epoch 59
Loss = 1.3679e-01, PNorm = 69.1455, GNorm = 4.3477, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.602856
Epoch 60
Validation binary_cross_entropy = 0.589302
Epoch 61
Validation binary_cross_entropy = 0.574561
Epoch 62
Validation binary_cross_entropy = 0.590163
Epoch 63
Validation binary_cross_entropy = 0.616740
Epoch 64
Loss = 2.8023e-02, PNorm = 69.2049, GNorm = 1.1845, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.636145
Epoch 65
Validation binary_cross_entropy = 0.627032
Epoch 66
Validation binary_cross_entropy = 0.630056
Epoch 67
Validation binary_cross_entropy = 0.657001
Epoch 68
Validation binary_cross_entropy = 0.675647
Epoch 69
Loss = 2.8628e-02, PNorm = 69.2653, GNorm = 1.4238, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.683335
Epoch 70
Validation binary_cross_entropy = 0.643619
Epoch 71
Validation binary_cross_entropy = 0.623663
Epoch 72
Validation binary_cross_entropy = 0.636234
Epoch 73
Validation binary_cross_entropy = 0.669980
Epoch 74
Loss = 1.6162e-01, PNorm = 69.3144, GNorm = 5.7491, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.655122
Epoch 75
Validation binary_cross_entropy = 0.684980
Epoch 76
Validation binary_cross_entropy = 0.726758
Epoch 77
Validation binary_cross_entropy = 0.693018
Epoch 78
Validation binary_cross_entropy = 0.652590
Epoch 79
Loss = 6.1120e-02, PNorm = 69.3691, GNorm = 1.6456, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.640044
Epoch 80
Validation binary_cross_entropy = 0.656380
Epoch 81
Validation binary_cross_entropy = 0.691459
Epoch 82
Validation binary_cross_entropy = 0.653690
Epoch 83
Validation binary_cross_entropy = 0.651681
Epoch 84
Loss = 1.7203e-01, PNorm = 69.4253, GNorm = 7.3356, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.630817
Epoch 85
Validation binary_cross_entropy = 0.722437
Epoch 86
Validation binary_cross_entropy = 0.673413
Epoch 87
Validation binary_cross_entropy = 0.610772
Epoch 88
Validation binary_cross_entropy = 0.629603
Epoch 89
Loss = 2.8777e-02, PNorm = 69.4861, GNorm = 1.8460, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.634341
Epoch 90
Validation binary_cross_entropy = 0.701739
Epoch 91
Validation binary_cross_entropy = 0.746116
Epoch 92
Validation binary_cross_entropy = 0.718673
Epoch 93
Validation binary_cross_entropy = 0.647752
Epoch 94
Loss = 4.6846e-02, PNorm = 69.5500, GNorm = 2.5737, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.686353
Epoch 95
Validation binary_cross_entropy = 0.756420
Epoch 96
Validation binary_cross_entropy = 0.645410
Epoch 97
Validation binary_cross_entropy = 0.776457
Epoch 98
Validation binary_cross_entropy = 0.910046
Epoch 99
Loss = 1.6298e-01, PNorm = 69.6224, GNorm = 7.4709, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.797990
Epoch 100
Validation binary_cross_entropy = 0.722550
Epoch 101
Validation binary_cross_entropy = 0.941765
Epoch 102
Validation binary_cross_entropy = 0.956759
Epoch 103
Validation binary_cross_entropy = 0.824150
Epoch 104
Loss = 1.0659e-01, PNorm = 69.7169, GNorm = 1.9100, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.743488
Epoch 105
Validation binary_cross_entropy = 0.691345
Epoch 106
Validation binary_cross_entropy = 0.672509
Epoch 107
Validation binary_cross_entropy = 0.681396
Epoch 108
Validation binary_cross_entropy = 0.687167
Epoch 109
Loss = 2.6002e-02, PNorm = 69.8114, GNorm = 0.6965, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.683177
Epoch 110
Validation binary_cross_entropy = 0.645358
Epoch 111
Validation binary_cross_entropy = 0.644023
Epoch 112
Validation binary_cross_entropy = 0.674798
Epoch 113
Validation binary_cross_entropy = 0.692376
Epoch 114
Loss = 4.7245e-02, PNorm = 69.8855, GNorm = 0.5608, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.699417
Epoch 115
Validation binary_cross_entropy = 0.765345
Epoch 116
Validation binary_cross_entropy = 0.829595
Epoch 117
Validation binary_cross_entropy = 0.788190
Epoch 118
Validation binary_cross_entropy = 0.716464
Epoch 119
Loss = 1.2182e-02, PNorm = 69.9779, GNorm = 0.3359, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.679980
Epoch 120
Validation binary_cross_entropy = 0.686301
Epoch 121
Validation binary_cross_entropy = 0.728226
Epoch 122
Validation binary_cross_entropy = 0.759708
Epoch 123
Validation binary_cross_entropy = 0.766388
Epoch 124
Loss = 4.6623e-02, PNorm = 70.0805, GNorm = 0.2668, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.774448
Epoch 125
Validation binary_cross_entropy = 0.766740
Epoch 126
Validation binary_cross_entropy = 0.762067
Epoch 127
Validation binary_cross_entropy = 0.754452
Epoch 128
Validation binary_cross_entropy = 0.764942
Epoch 129
Loss = 5.8030e-03, PNorm = 70.1590, GNorm = 0.3446, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.766821
Epoch 130
Validation binary_cross_entropy = 0.788773
Epoch 131
Validation binary_cross_entropy = 0.788953
Epoch 132
Validation binary_cross_entropy = 0.786748
Epoch 133
Validation binary_cross_entropy = 0.739346
Epoch 134
Loss = 1.9904e-02, PNorm = 70.2161, GNorm = 1.2910, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.714006
Epoch 135
Validation binary_cross_entropy = 0.716470
Epoch 136
Validation binary_cross_entropy = 0.753825
Epoch 137
Validation binary_cross_entropy = 0.789331
Epoch 138
Validation binary_cross_entropy = 0.796430
Epoch 139
Loss = 1.6442e-02, PNorm = 70.2480, GNorm = 1.5236, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.771143
Epoch 140
Validation binary_cross_entropy = 0.727461
Epoch 141
Validation binary_cross_entropy = 0.700943
Epoch 142
Validation binary_cross_entropy = 0.694335
Epoch 143
Validation binary_cross_entropy = 0.708457
Epoch 144
Loss = 7.2931e-03, PNorm = 70.2926, GNorm = 0.5017, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.741833
Epoch 145
Validation binary_cross_entropy = 0.778732
Epoch 146
Validation binary_cross_entropy = 0.801941
Epoch 147
Validation binary_cross_entropy = 0.802100
Epoch 148
Validation binary_cross_entropy = 0.786223
Epoch 149
Loss = 5.1684e-03, PNorm = 70.3306, GNorm = 0.2179, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.770250
Epoch 150
Validation binary_cross_entropy = 0.753667
Epoch 151
Validation binary_cross_entropy = 0.738573
Epoch 152
Validation binary_cross_entropy = 0.737975
Epoch 153
Validation binary_cross_entropy = 0.759812
Epoch 154
Loss = 3.8753e-03, PNorm = 70.3638, GNorm = 0.1027, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.786570
Epoch 155
Validation binary_cross_entropy = 0.803061
Epoch 156
Validation binary_cross_entropy = 0.806697
Epoch 157
Validation binary_cross_entropy = 0.798790
Epoch 158
Validation binary_cross_entropy = 0.787966
Epoch 159
Loss = 3.6764e-03, PNorm = 70.3906, GNorm = 0.1421, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.775695
Epoch 160
Validation binary_cross_entropy = 0.764879
Epoch 161
Validation binary_cross_entropy = 0.786967
Epoch 162
Validation binary_cross_entropy = 0.816503
Epoch 163
Validation binary_cross_entropy = 0.841622
Epoch 164
Loss = 9.0191e-03, PNorm = 70.4183, GNorm = 0.6937, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.842508
Epoch 165
Validation binary_cross_entropy = 0.804970
Epoch 166
Validation binary_cross_entropy = 0.777779
Epoch 167
Validation binary_cross_entropy = 0.764204
Epoch 168
Validation binary_cross_entropy = 0.759930
Epoch 169
Loss = 2.1486e-02, PNorm = 70.4480, GNorm = 3.1172, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.778348
Epoch 170
Validation binary_cross_entropy = 0.811425
Epoch 171
Validation binary_cross_entropy = 0.827529
Epoch 172
Validation binary_cross_entropy = 0.822641
Epoch 173
Validation binary_cross_entropy = 0.779247
Epoch 174
Loss = 3.6565e-03, PNorm = 70.4848, GNorm = 0.1767, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.754757
Epoch 175
Validation binary_cross_entropy = 0.747192
Epoch 176
Validation binary_cross_entropy = 0.757480
Epoch 177
Validation binary_cross_entropy = 0.789722
Epoch 178
Validation binary_cross_entropy = 0.827131
Epoch 179
Loss = 3.4510e-03, PNorm = 70.5095, GNorm = 0.0991, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.859384
Epoch 180
Validation binary_cross_entropy = 0.883117
Epoch 181
Validation binary_cross_entropy = 0.865730
Epoch 182
Validation binary_cross_entropy = 0.830259
Epoch 183
Validation binary_cross_entropy = 0.804710
Epoch 184
Loss = 5.9613e-02, PNorm = 70.5519, GNorm = 0.0537, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.846137
Epoch 185
Validation binary_cross_entropy = 0.881635
Epoch 186
Validation binary_cross_entropy = 0.913522
Epoch 187
Validation binary_cross_entropy = 0.871198
Epoch 188
Validation binary_cross_entropy = 0.815836
Epoch 189
Loss = 5.6856e-03, PNorm = 70.6019, GNorm = 0.0780, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.774278
Epoch 190
Validation binary_cross_entropy = 0.752083
Epoch 191
Validation binary_cross_entropy = 0.744513
Epoch 192
Validation binary_cross_entropy = 0.756647
Epoch 193
Validation binary_cross_entropy = 0.783581
Epoch 194
Loss = 1.7142e-02, PNorm = 70.6343, GNorm = 0.0775, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.796088
Epoch 195
Validation binary_cross_entropy = 0.801784
Epoch 196
Validation binary_cross_entropy = 0.805501
Epoch 197
Validation binary_cross_entropy = 0.845654
Epoch 198
Validation binary_cross_entropy = 0.873945
Epoch 199
Loss = 9.4626e-03, PNorm = 70.6507, GNorm = 0.0809, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.867805
Epoch 200
Validation binary_cross_entropy = 0.846543
Epoch 201
Validation binary_cross_entropy = 0.813934
Epoch 202
Validation binary_cross_entropy = 0.792173
Epoch 203
Validation binary_cross_entropy = 0.783584
Epoch 204
Loss = 7.9577e-03, PNorm = 70.6775, GNorm = 1.1286, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.789132
Epoch 205
Validation binary_cross_entropy = 0.812421
Epoch 206
Validation binary_cross_entropy = 0.840666
Epoch 207
Validation binary_cross_entropy = 0.922468
Epoch 208
Validation binary_cross_entropy = 0.994887
Epoch 209
Loss = 5.2850e-03, PNorm = 70.7070, GNorm = 0.3556, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.020869
Epoch 210
Validation binary_cross_entropy = 0.979112
Epoch 211
Validation binary_cross_entropy = 0.885324
Epoch 212
Validation binary_cross_entropy = 0.833238
Epoch 213
Validation binary_cross_entropy = 0.812778
Epoch 214
Loss = 1.5243e-02, PNorm = 70.7470, GNorm = 0.9134, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.816240
Epoch 215
Validation binary_cross_entropy = 0.851943
Epoch 216
Validation binary_cross_entropy = 0.939277
Epoch 217
Validation binary_cross_entropy = 1.002657
Epoch 218
Validation binary_cross_entropy = 0.963224
Epoch 219
Loss = 4.7827e-02, PNorm = 70.8096, GNorm = 0.1884, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.849802
Epoch 220
Validation binary_cross_entropy = 0.789779
Epoch 221
Validation binary_cross_entropy = 0.781895
Epoch 222
Validation binary_cross_entropy = 0.798534
Epoch 223
Validation binary_cross_entropy = 0.824354
Epoch 224
Loss = 5.6785e-03, PNorm = 70.8992, GNorm = 0.0909, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.834860
Epoch 225
Validation binary_cross_entropy = 0.809031
Epoch 226
Validation binary_cross_entropy = 0.772674
Epoch 227
Validation binary_cross_entropy = 0.777434
Epoch 228
Validation binary_cross_entropy = 0.865274
Epoch 229
Loss = 2.9504e-02, PNorm = 71.0440, GNorm = 0.1689, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.848272
Epoch 230
Validation binary_cross_entropy = 0.817038
Epoch 231
Validation binary_cross_entropy = 0.794279
Epoch 232
Validation binary_cross_entropy = 0.798298
Epoch 233
Validation binary_cross_entropy = 0.818236
Epoch 234
Loss = 2.3034e-02, PNorm = 71.3624, GNorm = 1.4725, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.842921
Epoch 235
Validation binary_cross_entropy = 0.822365
Epoch 236
Validation binary_cross_entropy = 0.810891
Epoch 237
Validation binary_cross_entropy = 0.807247
Epoch 238
Validation binary_cross_entropy = 0.819585
Epoch 239
Loss = 2.6867e-03, PNorm = 71.5893, GNorm = 0.0257, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.818034
Epoch 240
Validation binary_cross_entropy = 0.847154
Epoch 241
Validation binary_cross_entropy = 0.841999
Epoch 242
Validation binary_cross_entropy = 0.826878
Epoch 243
Validation binary_cross_entropy = 0.813504
Epoch 244
Loss = 6.2035e-03, PNorm = 71.7152, GNorm = 0.6826, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816158
Epoch 245
Validation binary_cross_entropy = 0.824030
Epoch 246
Validation binary_cross_entropy = 0.833417
Epoch 247
Validation binary_cross_entropy = 0.840339
Epoch 248
Validation binary_cross_entropy = 0.817595
Epoch 249
Loss = 3.3488e-03, PNorm = 71.7851, GNorm = 0.3124, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.815445
Epoch 250
Validation binary_cross_entropy = 0.832085
Epoch 251
Validation binary_cross_entropy = 0.892590
Epoch 252
Validation binary_cross_entropy = 0.922412
Epoch 253
Validation binary_cross_entropy = 0.924956
Epoch 254
Loss = 1.3270e-02, PNorm = 71.8291, GNorm = 2.3255, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.895851
Epoch 255
Validation binary_cross_entropy = 0.859924
Epoch 256
Validation binary_cross_entropy = 0.835751
Epoch 257
Validation binary_cross_entropy = 0.820376
Epoch 258
Validation binary_cross_entropy = 0.819261
Epoch 259
Loss = 1.8268e-03, PNorm = 71.8584, GNorm = 0.0326, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.827894
Epoch 260
Validation binary_cross_entropy = 0.841960
Epoch 261
Validation binary_cross_entropy = 0.854838
Epoch 262
Validation binary_cross_entropy = 0.867993
Epoch 263
Validation binary_cross_entropy = 0.876934
Epoch 264
Loss = 1.4528e-03, PNorm = 71.8797, GNorm = 0.1526, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.879753
Epoch 265
Validation binary_cross_entropy = 0.881254
Epoch 266
Validation binary_cross_entropy = 0.874664
Epoch 267
Validation binary_cross_entropy = 0.870089
Epoch 268
Validation binary_cross_entropy = 0.876220
Epoch 269
Loss = 1.4546e-03, PNorm = 71.8934, GNorm = 0.0391, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.886755
Epoch 270
Validation binary_cross_entropy = 0.896774
Epoch 271
Validation binary_cross_entropy = 0.906908
Epoch 272
Validation binary_cross_entropy = 0.912028
Epoch 273
Validation binary_cross_entropy = 0.913994
Epoch 274
Loss = 1.4120e-03, PNorm = 71.9044, GNorm = 0.0267, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.906616
Epoch 275
Validation binary_cross_entropy = 0.899282
Epoch 276
Validation binary_cross_entropy = 0.907030
Epoch 277
Validation binary_cross_entropy = 0.922115
Epoch 278
Validation binary_cross_entropy = 0.930425
Epoch 279
Loss = 2.6172e-02, PNorm = 71.9193, GNorm = 0.0687, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.874697
Epoch 280
Validation binary_cross_entropy = 0.855426
Epoch 281
Validation binary_cross_entropy = 0.880980
Epoch 282
Validation binary_cross_entropy = 0.904091
Epoch 283
Validation binary_cross_entropy = 0.924478
Epoch 284
Loss = 7.3554e-04, PNorm = 71.9468, GNorm = 0.0233, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.942037
Epoch 285
Validation binary_cross_entropy = 0.926044
Epoch 286
Validation binary_cross_entropy = 0.870195
Epoch 287
Validation binary_cross_entropy = 0.834073
Epoch 288
Validation binary_cross_entropy = 0.814292
Epoch 289
Loss = 3.7116e-03, PNorm = 71.9719, GNorm = 0.3142, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.809732
Epoch 290
Validation binary_cross_entropy = 0.822687
Epoch 291
Validation binary_cross_entropy = 0.842465
Epoch 292
Validation binary_cross_entropy = 0.866517
Epoch 293
Validation binary_cross_entropy = 0.909766
Epoch 294
Loss = 2.7880e-03, PNorm = 71.9994, GNorm = 0.0571, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.921037
Epoch 295
Validation binary_cross_entropy = 0.925828
Epoch 296
Validation binary_cross_entropy = 0.921880
Epoch 297
Validation binary_cross_entropy = 0.890025
Epoch 298
Validation binary_cross_entropy = 0.870897
Epoch 299
Loss = 1.1971e-03, PNorm = 72.0111, GNorm = 0.0395, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.856410
Model 0 best validation binary_cross_entropy = 0.443853 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223997
Ensemble test binary_cross_entropy = 0.223997
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.832462
Epoch 1
Validation binary_cross_entropy = 0.378559
Epoch 2
Validation binary_cross_entropy = 1.299092
Epoch 3
Validation binary_cross_entropy = 0.492696
Epoch 4
Loss = 6.1147e-01, PNorm = 68.1456, GNorm = 12.2124, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.407357
Epoch 5
Validation binary_cross_entropy = 0.912242
Epoch 6
Validation binary_cross_entropy = 0.443910
Epoch 7
Validation binary_cross_entropy = 0.516113
Epoch 8
Validation binary_cross_entropy = 0.971305
Epoch 9
Loss = 3.8370e-01, PNorm = 68.2994, GNorm = 9.0929, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.466086
Epoch 10
Validation binary_cross_entropy = 0.549780
Epoch 11
Validation binary_cross_entropy = 0.766453
Epoch 12
Validation binary_cross_entropy = 0.640310
Epoch 13
Validation binary_cross_entropy = 0.600659
Epoch 14
Loss = 3.7966e-01, PNorm = 68.4676, GNorm = 5.9524, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.540832
Epoch 15
Validation binary_cross_entropy = 0.692572
Epoch 16
Validation binary_cross_entropy = 0.647260
Epoch 17
Validation binary_cross_entropy = 0.566112
Epoch 18
Validation binary_cross_entropy = 0.500812
Epoch 19
Loss = 4.2456e-01, PNorm = 68.5997, GNorm = 5.5156, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.501260
Epoch 20
Validation binary_cross_entropy = 0.474353
Epoch 21
Validation binary_cross_entropy = 0.509510
Epoch 22
Validation binary_cross_entropy = 0.577811
Epoch 23
Validation binary_cross_entropy = 0.622608
Epoch 24
Loss = 9.3341e-02, PNorm = 68.6968, GNorm = 3.3987, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.638799
Epoch 25
Validation binary_cross_entropy = 0.658104
Epoch 26
Validation binary_cross_entropy = 0.591188
Epoch 27
Validation binary_cross_entropy = 0.535649
Epoch 28
Validation binary_cross_entropy = 0.510981
Epoch 29
Loss = 7.5652e-02, PNorm = 68.7694, GNorm = 2.5995, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.516488
Epoch 30
Validation binary_cross_entropy = 0.547929
Epoch 31
Validation binary_cross_entropy = 0.579443
Epoch 32
Validation binary_cross_entropy = 0.603359
Epoch 33
Validation binary_cross_entropy = 0.621219
Epoch 34
Loss = 5.0912e-02, PNorm = 68.8334, GNorm = 0.7971, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.630961
Epoch 35
Validation binary_cross_entropy = 0.626331
Epoch 36
Validation binary_cross_entropy = 0.612054
Epoch 37
Validation binary_cross_entropy = 0.658149
Epoch 38
Validation binary_cross_entropy = 0.570956
Epoch 39
Loss = 1.1161e-01, PNorm = 68.9066, GNorm = 3.3734, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.576933
Epoch 40
Validation binary_cross_entropy = 0.606871
Epoch 41
Validation binary_cross_entropy = 0.626640
Epoch 42
Validation binary_cross_entropy = 0.632824
Epoch 43
Validation binary_cross_entropy = 0.613828
Epoch 44
Loss = 8.0760e-02, PNorm = 68.9819, GNorm = 3.6395, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.600283
Epoch 45
Validation binary_cross_entropy = 0.567970
Epoch 46
Validation binary_cross_entropy = 0.679428
Epoch 47
Validation binary_cross_entropy = 0.575334
Epoch 48
Validation binary_cross_entropy = 0.638561
Epoch 49
Loss = 1.8453e-01, PNorm = 69.0651, GNorm = 5.0354, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.556683
Epoch 50
Validation binary_cross_entropy = 0.738229
Epoch 51
Validation binary_cross_entropy = 0.671503
Epoch 52
Validation binary_cross_entropy = 0.643175
Epoch 53
Validation binary_cross_entropy = 0.694153
Epoch 54
Loss = 3.2413e-02, PNorm = 69.1509, GNorm = 2.9867, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.688209
Epoch 55
Validation binary_cross_entropy = 0.699134
Epoch 56
Validation binary_cross_entropy = 0.648920
Epoch 57
Validation binary_cross_entropy = 0.632678
Epoch 58
Validation binary_cross_entropy = 0.628885
Epoch 59
Loss = 1.0407e-01, PNorm = 69.2198, GNorm = 2.0008, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.736914
Epoch 60
Validation binary_cross_entropy = 0.675786
Epoch 61
Validation binary_cross_entropy = 0.752781
Epoch 62
Validation binary_cross_entropy = 0.687824
Epoch 63
Validation binary_cross_entropy = 0.718905
Epoch 64
Loss = 4.4337e-02, PNorm = 69.2984, GNorm = 1.3956, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.907884
Epoch 65
Validation binary_cross_entropy = 0.761959
Epoch 66
Validation binary_cross_entropy = 0.805388
Epoch 67
Validation binary_cross_entropy = 0.794945
Epoch 68
Validation binary_cross_entropy = 0.763754
Epoch 69
Loss = 4.6166e-02, PNorm = 69.4266, GNorm = 1.2760, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.929151
Epoch 70
Validation binary_cross_entropy = 0.751765
Epoch 71
Validation binary_cross_entropy = 0.689020
Epoch 72
Validation binary_cross_entropy = 0.673602
Epoch 73
Validation binary_cross_entropy = 0.693447
Epoch 74
Loss = 1.4982e-01, PNorm = 69.5261, GNorm = 4.8765, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.662485
Epoch 75
Validation binary_cross_entropy = 0.657429
Epoch 76
Validation binary_cross_entropy = 0.664215
Epoch 77
Validation binary_cross_entropy = 0.704733
Epoch 78
Validation binary_cross_entropy = 0.719832
Epoch 79
Loss = 5.2371e-02, PNorm = 69.6051, GNorm = 1.4204, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.707854
Epoch 80
Validation binary_cross_entropy = 0.711265
Epoch 81
Validation binary_cross_entropy = 0.730334
Epoch 82
Validation binary_cross_entropy = 0.798368
Epoch 83
Validation binary_cross_entropy = 0.893634
Epoch 84
Loss = 1.4113e-01, PNorm = 69.6658, GNorm = 5.8820, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.790659
Epoch 85
Validation binary_cross_entropy = 0.817033
Epoch 86
Validation binary_cross_entropy = 0.774534
Epoch 87
Validation binary_cross_entropy = 0.784872
Epoch 88
Validation binary_cross_entropy = 0.879939
Epoch 89
Loss = 8.6250e-02, PNorm = 69.7251, GNorm = 2.8842, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.885941
Epoch 90
Validation binary_cross_entropy = 0.750530
Epoch 91
Validation binary_cross_entropy = 0.734723
Epoch 92
Validation binary_cross_entropy = 0.762397
Epoch 93
Validation binary_cross_entropy = 0.808543
Epoch 94
Loss = 4.9345e-02, PNorm = 69.7859, GNorm = 3.5747, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.812751
Epoch 95
Validation binary_cross_entropy = 0.795898
Epoch 96
Validation binary_cross_entropy = 0.797010
Epoch 97
Validation binary_cross_entropy = 0.799072
Epoch 98
Validation binary_cross_entropy = 0.811164
Epoch 99
Loss = 3.6332e-02, PNorm = 69.8355, GNorm = 0.8393, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.823658
Epoch 100
Validation binary_cross_entropy = 0.798788
Epoch 101
Validation binary_cross_entropy = 0.724793
Epoch 102
Validation binary_cross_entropy = 0.688816
Epoch 103
Validation binary_cross_entropy = 0.700114
Epoch 104
Loss = 1.0036e-01, PNorm = 69.8806, GNorm = 5.8868, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.728743
Epoch 105
Validation binary_cross_entropy = 0.848734
Epoch 106
Validation binary_cross_entropy = 0.868268
Epoch 107
Validation binary_cross_entropy = 0.784426
Epoch 108
Validation binary_cross_entropy = 0.764476
Epoch 109
Loss = 4.1696e-02, PNorm = 69.9303, GNorm = 2.2748, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.754812
Epoch 110
Validation binary_cross_entropy = 0.770340
Epoch 111
Validation binary_cross_entropy = 0.794510
Epoch 112
Validation binary_cross_entropy = 0.774802
Epoch 113
Validation binary_cross_entropy = 0.790034
Epoch 114
Loss = 8.6346e-03, PNorm = 69.9805, GNorm = 0.8543, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.820064
Epoch 115
Validation binary_cross_entropy = 0.823312
Epoch 116
Validation binary_cross_entropy = 0.792664
Epoch 117
Validation binary_cross_entropy = 0.761983
Epoch 118
Validation binary_cross_entropy = 0.736014
Epoch 119
Loss = 2.5750e-02, PNorm = 70.0273, GNorm = 0.7028, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.753033
Epoch 120
Validation binary_cross_entropy = 0.790966
Epoch 121
Validation binary_cross_entropy = 0.833668
Epoch 122
Validation binary_cross_entropy = 0.822120
Epoch 123
Validation binary_cross_entropy = 0.759804
Epoch 124
Loss = 4.5020e-03, PNorm = 70.0638, GNorm = 0.1544, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.712759
Epoch 125
Validation binary_cross_entropy = 0.690907
Epoch 126
Validation binary_cross_entropy = 0.684080
Epoch 127
Validation binary_cross_entropy = 0.701368
Epoch 128
Validation binary_cross_entropy = 0.728350
Epoch 129
Loss = 5.5843e-03, PNorm = 70.0926, GNorm = 0.2285, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.752393
Epoch 130
Validation binary_cross_entropy = 0.804121
Epoch 131
Validation binary_cross_entropy = 0.868056
Epoch 132
Validation binary_cross_entropy = 0.870424
Epoch 133
Validation binary_cross_entropy = 0.827046
Epoch 134
Loss = 6.1280e-03, PNorm = 70.1203, GNorm = 0.4157, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.777622
Epoch 135
Validation binary_cross_entropy = 0.742025
Epoch 136
Validation binary_cross_entropy = 0.734854
Epoch 137
Validation binary_cross_entropy = 0.741393
Epoch 138
Validation binary_cross_entropy = 0.756611
Epoch 139
Loss = 3.7221e-03, PNorm = 70.1473, GNorm = 0.1789, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.772355
Epoch 140
Validation binary_cross_entropy = 0.787818
Epoch 141
Validation binary_cross_entropy = 0.798850
Epoch 142
Validation binary_cross_entropy = 0.807385
Epoch 143
Validation binary_cross_entropy = 0.863579
Epoch 144
Loss = 7.6110e-02, PNorm = 70.1685, GNorm = 3.8632, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.869751
Epoch 145
Validation binary_cross_entropy = 0.816165
Epoch 146
Validation binary_cross_entropy = 0.778904
Epoch 147
Validation binary_cross_entropy = 0.767935
Epoch 148
Validation binary_cross_entropy = 0.762180
Epoch 149
Loss = 7.2083e-03, PNorm = 70.1919, GNorm = 0.9222, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.769814
Epoch 150
Validation binary_cross_entropy = 0.790532
Epoch 151
Validation binary_cross_entropy = 0.822447
Epoch 152
Validation binary_cross_entropy = 0.917815
Epoch 153
Validation binary_cross_entropy = 0.984877
Epoch 154
Loss = 3.9538e-02, PNorm = 70.2149, GNorm = 0.3629, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.924717
Epoch 155
Validation binary_cross_entropy = 0.812233
Epoch 156
Validation binary_cross_entropy = 0.795036
Epoch 157
Validation binary_cross_entropy = 0.805774
Epoch 158
Validation binary_cross_entropy = 0.819046
Epoch 159
Loss = 4.3592e-03, PNorm = 70.2466, GNorm = 0.2286, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.832247
Epoch 160
Validation binary_cross_entropy = 0.881124
Epoch 161
Validation binary_cross_entropy = 0.914741
Epoch 162
Validation binary_cross_entropy = 0.931620
Epoch 163
Validation binary_cross_entropy = 0.925658
Epoch 164
Loss = 1.1192e-02, PNorm = 70.2747, GNorm = 0.6754, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.876487
Epoch 165
Validation binary_cross_entropy = 0.815951
Epoch 166
Validation binary_cross_entropy = 0.792609
Epoch 167
Validation binary_cross_entropy = 0.791536
Epoch 168
Validation binary_cross_entropy = 0.808715
Epoch 169
Loss = 6.7676e-03, PNorm = 70.3047, GNorm = 0.6089, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.844731
Epoch 170
Validation binary_cross_entropy = 0.885662
Epoch 171
Validation binary_cross_entropy = 0.909296
Epoch 172
Validation binary_cross_entropy = 0.899417
Epoch 173
Validation binary_cross_entropy = 0.872273
Epoch 174
Loss = 4.1053e-03, PNorm = 70.3441, GNorm = 0.0741, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.855373
Epoch 175
Validation binary_cross_entropy = 0.832029
Epoch 176
Validation binary_cross_entropy = 0.843888
Epoch 177
Validation binary_cross_entropy = 0.853921
Epoch 178
Validation binary_cross_entropy = 0.859819
Epoch 179
Loss = 5.3328e-03, PNorm = 70.4086, GNorm = 0.7352, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.850426
Epoch 180
Validation binary_cross_entropy = 0.810188
Epoch 181
Validation binary_cross_entropy = 0.816852
Epoch 182
Validation binary_cross_entropy = 0.844919
Epoch 183
Validation binary_cross_entropy = 0.857477
Epoch 184
Loss = 6.6188e-02, PNorm = 70.4493, GNorm = 4.7918, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.828303
Epoch 185
Validation binary_cross_entropy = 0.797323
Epoch 186
Validation binary_cross_entropy = 0.819420
Epoch 187
Validation binary_cross_entropy = 0.868371
Epoch 188
Validation binary_cross_entropy = 0.923063
Epoch 189
Loss = 4.1317e-03, PNorm = 70.5605, GNorm = 0.1895, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.968290
Epoch 190
Validation binary_cross_entropy = 0.952176
Epoch 191
Validation binary_cross_entropy = 0.915546
Epoch 192
Validation binary_cross_entropy = 0.903760
Epoch 193
Validation binary_cross_entropy = 0.931497
Epoch 194
Loss = 1.4975e-02, PNorm = 70.6228, GNorm = 1.7455, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.925423
Epoch 195
Validation binary_cross_entropy = 0.896076
Epoch 196
Validation binary_cross_entropy = 0.874537
Epoch 197
Validation binary_cross_entropy = 0.824678
Epoch 198
Validation binary_cross_entropy = 0.796779
Epoch 199
Loss = 3.4465e-02, PNorm = 70.6644, GNorm = 2.8352, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.767295
Epoch 200
Validation binary_cross_entropy = 0.790872
Epoch 201
Validation binary_cross_entropy = 0.897977
Epoch 202
Validation binary_cross_entropy = 0.971248
Epoch 203
Validation binary_cross_entropy = 0.959522
Epoch 204
Loss = 4.0764e-03, PNorm = 70.7105, GNorm = 0.0867, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.921456
Epoch 205
Validation binary_cross_entropy = 0.872928
Epoch 206
Validation binary_cross_entropy = 0.833842
Epoch 207
Validation binary_cross_entropy = 0.817037
Epoch 208
Validation binary_cross_entropy = 0.811077
Epoch 209
Loss = 7.5920e-03, PNorm = 70.7405, GNorm = 0.2331, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.799208
Epoch 210
Validation binary_cross_entropy = 0.803429
Epoch 211
Validation binary_cross_entropy = 0.812728
Epoch 212
Validation binary_cross_entropy = 0.804286
Epoch 213
Validation binary_cross_entropy = 0.835610
Epoch 214
Loss = 2.2437e-03, PNorm = 70.7741, GNorm = 0.1275, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.870157
Epoch 215
Validation binary_cross_entropy = 0.896534
Epoch 216
Validation binary_cross_entropy = 0.906554
Epoch 217
Validation binary_cross_entropy = 0.884549
Epoch 218
Validation binary_cross_entropy = 0.888453
Epoch 219
Loss = 1.5597e-03, PNorm = 70.8139, GNorm = 0.0239, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.918318
Epoch 220
Validation binary_cross_entropy = 0.935118
Epoch 221
Validation binary_cross_entropy = 0.909114
Epoch 222
Validation binary_cross_entropy = 0.862354
Epoch 223
Validation binary_cross_entropy = 0.834052
Epoch 224
Loss = 4.8088e-03, PNorm = 70.8414, GNorm = 0.4314, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.820897
Epoch 225
Validation binary_cross_entropy = 0.816335
Epoch 226
Validation binary_cross_entropy = 0.807667
Epoch 227
Validation binary_cross_entropy = 0.861008
Epoch 228
Validation binary_cross_entropy = 0.931418
Epoch 229
Loss = 1.1936e-02, PNorm = 70.8863, GNorm = 1.5049, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.967685
Epoch 230
Validation binary_cross_entropy = 0.933780
Epoch 231
Validation binary_cross_entropy = 0.906862
Epoch 232
Validation binary_cross_entropy = 0.874766
Epoch 233
Validation binary_cross_entropy = 0.868301
Epoch 234
Loss = 3.0030e-02, PNorm = 70.9836, GNorm = 1.9606, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.881167
Epoch 235
Validation binary_cross_entropy = 0.916198
Epoch 236
Validation binary_cross_entropy = 0.939828
Epoch 237
Validation binary_cross_entropy = 0.940282
Epoch 238
Validation binary_cross_entropy = 0.906809
Epoch 239
Loss = 1.9161e-03, PNorm = 71.0690, GNorm = 0.0738, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.881236
Epoch 240
Validation binary_cross_entropy = 0.869710
Epoch 241
Validation binary_cross_entropy = 0.889762
Epoch 242
Validation binary_cross_entropy = 0.905806
Epoch 243
Validation binary_cross_entropy = 0.914320
Epoch 244
Loss = 9.5510e-02, PNorm = 71.1551, GNorm = 3.5345, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.951135
Epoch 245
Validation binary_cross_entropy = 1.022378
Epoch 246
Validation binary_cross_entropy = 0.999768
Epoch 247
Validation binary_cross_entropy = 0.893496
Epoch 248
Validation binary_cross_entropy = 0.853296
Epoch 249
Loss = 1.9221e-03, PNorm = 71.2111, GNorm = 0.1001, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.827395
Epoch 250
Validation binary_cross_entropy = 0.816999
Epoch 251
Validation binary_cross_entropy = 0.813352
Epoch 252
Validation binary_cross_entropy = 0.817014
Epoch 253
Validation binary_cross_entropy = 0.828168
Epoch 254
Loss = 4.8609e-03, PNorm = 71.2619, GNorm = 0.2632, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.850426
Epoch 255
Validation binary_cross_entropy = 0.874675
Epoch 256
Validation binary_cross_entropy = 0.898376
Epoch 257
Validation binary_cross_entropy = 0.927420
Epoch 258
Validation binary_cross_entropy = 0.956614
Epoch 259
Loss = 2.6151e-03, PNorm = 71.2922, GNorm = 0.0255, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.968479
Epoch 260
Validation binary_cross_entropy = 0.976226
Epoch 261
Validation binary_cross_entropy = 0.977371
Epoch 262
Validation binary_cross_entropy = 0.968671
Epoch 263
Validation binary_cross_entropy = 0.953196
Epoch 264
Loss = 2.1781e-03, PNorm = 71.3126, GNorm = 0.1375, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.934552
Epoch 265
Validation binary_cross_entropy = 0.915205
Epoch 266
Validation binary_cross_entropy = 0.895664
Epoch 267
Validation binary_cross_entropy = 0.879681
Epoch 268
Validation binary_cross_entropy = 0.866114
Epoch 269
Loss = 1.4389e-03, PNorm = 71.3257, GNorm = 0.0536, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.858087
Epoch 270
Validation binary_cross_entropy = 0.857844
Epoch 271
Validation binary_cross_entropy = 0.862540
Epoch 272
Validation binary_cross_entropy = 0.869681
Epoch 273
Validation binary_cross_entropy = 0.877668
Epoch 274
Loss = 2.5038e-03, PNorm = 71.3334, GNorm = 0.1285, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.888372
Epoch 275
Validation binary_cross_entropy = 0.893839
Epoch 276
Validation binary_cross_entropy = 0.888840
Epoch 277
Validation binary_cross_entropy = 0.876890
Epoch 278
Validation binary_cross_entropy = 0.866871
Epoch 279
Loss = 6.0368e-04, PNorm = 71.3686, GNorm = 0.0188, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.858824
Epoch 280
Validation binary_cross_entropy = 0.905212
Epoch 281
Validation binary_cross_entropy = 0.945286
Epoch 282
Validation binary_cross_entropy = 0.920307
Epoch 283
Validation binary_cross_entropy = 0.897882
Epoch 284
Loss = 2.3283e-03, PNorm = 71.3945, GNorm = 0.3768, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.879069
Epoch 285
Validation binary_cross_entropy = 0.866034
Epoch 286
Validation binary_cross_entropy = 0.858369
Epoch 287
Validation binary_cross_entropy = 0.853755
Epoch 288
Validation binary_cross_entropy = 0.851130
Epoch 289
Loss = 6.6658e-04, PNorm = 71.4089, GNorm = 0.0426, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.850105
Epoch 290
Validation binary_cross_entropy = 0.853810
Epoch 291
Validation binary_cross_entropy = 0.864588
Epoch 292
Validation binary_cross_entropy = 0.875986
Epoch 293
Validation binary_cross_entropy = 0.891414
Epoch 294
Loss = 9.7052e-04, PNorm = 71.4170, GNorm = 0.0448, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.907425
Epoch 295
Validation binary_cross_entropy = 0.919373
Epoch 296
Validation binary_cross_entropy = 0.926611
Epoch 297
Validation binary_cross_entropy = 0.963497
Epoch 298
Validation binary_cross_entropy = 1.023532
Epoch 299
Loss = 3.9703e-03, PNorm = 71.4284, GNorm = 0.5106, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.057407
Model 0 best validation binary_cross_entropy = 0.378559 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.182930
Ensemble test binary_cross_entropy = 0.182930
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.548277
Epoch 1
Validation binary_cross_entropy = 0.498330
Epoch 2
Validation binary_cross_entropy = 0.936247
Epoch 3
Validation binary_cross_entropy = 0.371793
Epoch 4
Loss = 5.4634e-01, PNorm = 68.1508, GNorm = 6.9759, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.145972
Epoch 5
Validation binary_cross_entropy = 0.717411
Epoch 6
Validation binary_cross_entropy = 0.491141
Epoch 7
Validation binary_cross_entropy = 0.950882
Epoch 8
Validation binary_cross_entropy = 0.528130
Epoch 9
Loss = 4.7522e-01, PNorm = 68.3112, GNorm = 9.2699, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.524242
Epoch 10
Validation binary_cross_entropy = 0.854400
Epoch 11
Validation binary_cross_entropy = 0.627289
Epoch 12
Validation binary_cross_entropy = 0.566897
Epoch 13
Validation binary_cross_entropy = 0.544316
Epoch 14
Loss = 2.7115e-01, PNorm = 68.4879, GNorm = 5.8185, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.648788
Epoch 15
Validation binary_cross_entropy = 0.721275
Epoch 16
Validation binary_cross_entropy = 1.044241
Epoch 17
Validation binary_cross_entropy = 0.769371
Epoch 18
Validation binary_cross_entropy = 0.747489
Epoch 19
Loss = 2.2142e-01, PNorm = 68.6147, GNorm = 5.5396, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.703936
Epoch 20
Validation binary_cross_entropy = 0.687927
Epoch 21
Validation binary_cross_entropy = 0.584586
Epoch 22
Validation binary_cross_entropy = 0.540043
Epoch 23
Validation binary_cross_entropy = 0.541269
Epoch 24
Loss = 8.3795e-02, PNorm = 68.7059, GNorm = 2.5651, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.583353
Epoch 25
Validation binary_cross_entropy = 0.563689
Epoch 26
Validation binary_cross_entropy = 0.624886
Epoch 27
Validation binary_cross_entropy = 0.602985
Epoch 28
Validation binary_cross_entropy = 0.587039
Epoch 29
Loss = 1.9182e-01, PNorm = 68.7706, GNorm = 4.6895, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.626662
Epoch 30
Validation binary_cross_entropy = 0.624202
Epoch 31
Validation binary_cross_entropy = 0.570385
Epoch 32
Validation binary_cross_entropy = 0.523552
Epoch 33
Validation binary_cross_entropy = 0.567745
Epoch 34
Loss = 1.1192e-01, PNorm = 68.8420, GNorm = 2.5080, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.565583
Epoch 35
Validation binary_cross_entropy = 0.563759
Epoch 36
Validation binary_cross_entropy = 0.581738
Epoch 37
Validation binary_cross_entropy = 0.716679
Epoch 38
Validation binary_cross_entropy = 0.703806
Epoch 39
Loss = 6.4467e-02, PNorm = 68.9175, GNorm = 1.9730, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.703850
Epoch 40
Validation binary_cross_entropy = 0.749610
Epoch 41
Validation binary_cross_entropy = 0.690204
Epoch 42
Validation binary_cross_entropy = 0.697750
Epoch 43
Validation binary_cross_entropy = 0.712077
Epoch 44
Loss = 1.6319e-01, PNorm = 69.0046, GNorm = 7.9030, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.727623
Epoch 45
Validation binary_cross_entropy = 0.803087
Epoch 46
Validation binary_cross_entropy = 0.765489
Epoch 47
Validation binary_cross_entropy = 0.691737
Epoch 48
Validation binary_cross_entropy = 0.679606
Epoch 49
Loss = 9.7964e-02, PNorm = 69.1352, GNorm = 4.6613, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.672899
Epoch 50
Validation binary_cross_entropy = 0.609115
Epoch 51
Validation binary_cross_entropy = 0.606457
Epoch 52
Validation binary_cross_entropy = 0.633137
Epoch 53
Validation binary_cross_entropy = 0.694805
Epoch 54
Loss = 8.2451e-02, PNorm = 69.2307, GNorm = 2.2450, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.674437
Epoch 55
Validation binary_cross_entropy = 0.661574
Epoch 56
Validation binary_cross_entropy = 0.661608
Epoch 57
Validation binary_cross_entropy = 0.707575
Epoch 58
Validation binary_cross_entropy = 0.731822
Epoch 59
Loss = 3.9618e-02, PNorm = 69.2937, GNorm = 0.9236, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.675070
Epoch 60
Validation binary_cross_entropy = 0.628326
Epoch 61
Validation binary_cross_entropy = 0.647492
Epoch 62
Validation binary_cross_entropy = 0.676269
Epoch 63
Validation binary_cross_entropy = 0.592893
Epoch 64
Loss = 1.3264e-01, PNorm = 69.3469, GNorm = 3.2643, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.607614
Epoch 65
Validation binary_cross_entropy = 0.668867
Epoch 66
Validation binary_cross_entropy = 0.762305
Epoch 67
Validation binary_cross_entropy = 0.858216
Epoch 68
Validation binary_cross_entropy = 0.797573
Epoch 69
Loss = 5.5612e-02, PNorm = 69.4010, GNorm = 2.4586, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.720866
Epoch 70
Validation binary_cross_entropy = 0.638527
Epoch 71
Validation binary_cross_entropy = 0.610892
Epoch 72
Validation binary_cross_entropy = 0.630817
Epoch 73
Validation binary_cross_entropy = 0.696876
Epoch 74
Loss = 3.0689e-02, PNorm = 69.4742, GNorm = 0.3597, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.740047
Epoch 75
Validation binary_cross_entropy = 0.776502
Epoch 76
Validation binary_cross_entropy = 0.762641
Epoch 77
Validation binary_cross_entropy = 0.733285
Epoch 78
Validation binary_cross_entropy = 0.703321
Epoch 79
Loss = 7.7547e-02, PNorm = 69.5273, GNorm = 0.2434, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.730784
Epoch 80
Validation binary_cross_entropy = 0.734506
Epoch 81
Validation binary_cross_entropy = 0.723035
Epoch 82
Validation binary_cross_entropy = 0.716474
Epoch 83
Validation binary_cross_entropy = 0.695076
Epoch 84
Loss = 9.9598e-03, PNorm = 69.6043, GNorm = 0.3204, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.689102
Epoch 85
Validation binary_cross_entropy = 0.708082
Epoch 86
Validation binary_cross_entropy = 0.798554
Epoch 87
Validation binary_cross_entropy = 0.873201
Epoch 88
Validation binary_cross_entropy = 0.809327
Epoch 89
Loss = 6.3801e-02, PNorm = 69.6777, GNorm = 2.0886, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.747776
Epoch 90
Validation binary_cross_entropy = 0.749161
Epoch 91
Validation binary_cross_entropy = 0.735027
Epoch 92
Validation binary_cross_entropy = 0.748254
Epoch 93
Validation binary_cross_entropy = 0.731866
Epoch 94
Loss = 2.4719e-02, PNorm = 69.7437, GNorm = 2.1533, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.683985
Epoch 95
Validation binary_cross_entropy = 0.670555
Epoch 96
Validation binary_cross_entropy = 0.677413
Epoch 97
Validation binary_cross_entropy = 0.682150
Epoch 98
Validation binary_cross_entropy = 0.739481
Epoch 99
Loss = 4.1913e-02, PNorm = 69.7962, GNorm = 4.7970, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.794425
Epoch 100
Validation binary_cross_entropy = 0.788945
Epoch 101
Validation binary_cross_entropy = 0.776087
Epoch 102
Validation binary_cross_entropy = 0.747286
Epoch 103
Validation binary_cross_entropy = 0.760806
Epoch 104
Loss = 9.1562e-03, PNorm = 69.8299, GNorm = 0.7049, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.772899
Epoch 105
Validation binary_cross_entropy = 0.769955
Epoch 106
Validation binary_cross_entropy = 0.749921
Epoch 107
Validation binary_cross_entropy = 0.722973
Epoch 108
Validation binary_cross_entropy = 0.720127
Epoch 109
Loss = 1.0634e-02, PNorm = 69.8625, GNorm = 0.8607, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.732026
Epoch 110
Validation binary_cross_entropy = 0.742911
Epoch 111
Validation binary_cross_entropy = 0.768567
Epoch 112
Validation binary_cross_entropy = 0.786528
Epoch 113
Validation binary_cross_entropy = 0.797053
Epoch 114
Loss = 8.0439e-03, PNorm = 69.8875, GNorm = 0.1892, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.790638
Epoch 115
Validation binary_cross_entropy = 0.785905
Epoch 116
Validation binary_cross_entropy = 0.781741
Epoch 117
Validation binary_cross_entropy = 0.779588
Epoch 118
Validation binary_cross_entropy = 0.786941
Epoch 119
Loss = 5.4476e-02, PNorm = 69.9103, GNorm = 1.0275, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.837001
Epoch 120
Validation binary_cross_entropy = 0.866938
Epoch 121
Validation binary_cross_entropy = 0.856589
Epoch 122
Validation binary_cross_entropy = 0.804123
Epoch 123
Validation binary_cross_entropy = 0.748548
Epoch 124
Loss = 2.9032e-02, PNorm = 69.9974, GNorm = 1.4944, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.749109
Epoch 125
Validation binary_cross_entropy = 0.775646
Epoch 126
Validation binary_cross_entropy = 0.807785
Epoch 127
Validation binary_cross_entropy = 0.856453
Epoch 128
Validation binary_cross_entropy = 0.839605
Epoch 129
Loss = 1.9518e-02, PNorm = 70.0579, GNorm = 1.3920, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.816344
Epoch 130
Validation binary_cross_entropy = 0.814422
Epoch 131
Validation binary_cross_entropy = 0.798938
Epoch 132
Validation binary_cross_entropy = 0.800515
Epoch 133
Validation binary_cross_entropy = 0.820492
Epoch 134
Loss = 2.1364e-02, PNorm = 70.1126, GNorm = 0.1528, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.867916
Epoch 135
Validation binary_cross_entropy = 0.898768
Epoch 136
Validation binary_cross_entropy = 0.894612
Epoch 137
Validation binary_cross_entropy = 0.922628
Epoch 138
Validation binary_cross_entropy = 0.904853
Epoch 139
Loss = 7.5496e-02, PNorm = 70.1573, GNorm = 4.5464, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.813735
Epoch 140
Validation binary_cross_entropy = 0.794656
Epoch 141
Validation binary_cross_entropy = 0.738580
Epoch 142
Validation binary_cross_entropy = 0.786305
Epoch 143
Validation binary_cross_entropy = 0.885700
Epoch 144
Loss = 2.6617e-02, PNorm = 70.2306, GNorm = 1.4902, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.884253
Epoch 145
Validation binary_cross_entropy = 0.796329
Epoch 146
Validation binary_cross_entropy = 0.756065
Epoch 147
Validation binary_cross_entropy = 0.767679
Epoch 148
Validation binary_cross_entropy = 0.787908
Epoch 149
Loss = 2.3158e-02, PNorm = 70.3081, GNorm = 0.0896, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.778720
Epoch 150
Validation binary_cross_entropy = 0.782315
Epoch 151
Validation binary_cross_entropy = 0.821738
Epoch 152
Validation binary_cross_entropy = 0.911679
Epoch 153
Validation binary_cross_entropy = 0.896632
Epoch 154
Loss = 1.5640e-02, PNorm = 70.3631, GNorm = 0.1079, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.793837
Epoch 155
Validation binary_cross_entropy = 0.766316
Epoch 156
Validation binary_cross_entropy = 0.773228
Epoch 157
Validation binary_cross_entropy = 0.803886
Epoch 158
Validation binary_cross_entropy = 0.826155
Epoch 159
Loss = 1.4070e-02, PNorm = 70.4177, GNorm = 2.5739, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.819451
Epoch 160
Validation binary_cross_entropy = 0.808560
Epoch 161
Validation binary_cross_entropy = 0.823062
Epoch 162
Validation binary_cross_entropy = 0.852557
Epoch 163
Validation binary_cross_entropy = 0.873205
Epoch 164
Loss = 2.3004e-02, PNorm = 70.4804, GNorm = 1.6942, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.900106
Epoch 165
Validation binary_cross_entropy = 0.931103
Epoch 166
Validation binary_cross_entropy = 0.938676
Epoch 167
Validation binary_cross_entropy = 0.919986
Epoch 168
Validation binary_cross_entropy = 0.907692
Epoch 169
Loss = 5.2771e-03, PNorm = 70.5282, GNorm = 0.7045, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.893613
Epoch 170
Validation binary_cross_entropy = 0.878593
Epoch 171
Validation binary_cross_entropy = 0.872803
Epoch 172
Validation binary_cross_entropy = 0.878099
Epoch 173
Validation binary_cross_entropy = 0.920478
Epoch 174
Loss = 5.7668e-03, PNorm = 70.5642, GNorm = 0.3827, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.976231
Epoch 175
Validation binary_cross_entropy = 0.975647
Epoch 176
Validation binary_cross_entropy = 0.944614
Epoch 177
Validation binary_cross_entropy = 0.908852
Epoch 178
Validation binary_cross_entropy = 0.881982
Epoch 179
Loss = 7.5204e-03, PNorm = 70.6507, GNorm = 0.1339, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.860982
Epoch 180
Validation binary_cross_entropy = 0.863287
Epoch 181
Validation binary_cross_entropy = 0.890919
Epoch 182
Validation binary_cross_entropy = 0.928628
Epoch 183
Validation binary_cross_entropy = 0.950007
Epoch 184
Loss = 9.0293e-03, PNorm = 70.7273, GNorm = 0.8371, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.953599
Epoch 185
Validation binary_cross_entropy = 0.923542
Epoch 186
Validation binary_cross_entropy = 0.900035
Epoch 187
Validation binary_cross_entropy = 0.885514
Epoch 188
Validation binary_cross_entropy = 0.878062
Epoch 189
Loss = 4.6094e-03, PNorm = 70.7581, GNorm = 0.0981, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.880901
Epoch 190
Validation binary_cross_entropy = 0.885597
Epoch 191
Validation binary_cross_entropy = 0.893551
Epoch 192
Validation binary_cross_entropy = 0.904232
Epoch 193
Validation binary_cross_entropy = 0.930805
Epoch 194
Loss = 4.6016e-03, PNorm = 70.7752, GNorm = 0.6721, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.942534
Epoch 195
Validation binary_cross_entropy = 0.941757
Epoch 196
Validation binary_cross_entropy = 0.926233
Epoch 197
Validation binary_cross_entropy = 0.904627
Epoch 198
Validation binary_cross_entropy = 0.871817
Epoch 199
Loss = 6.6842e-03, PNorm = 70.7921, GNorm = 0.0364, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.861055
Epoch 200
Validation binary_cross_entropy = 0.846282
Epoch 201
Validation binary_cross_entropy = 0.840961
Epoch 202
Validation binary_cross_entropy = 0.838746
Epoch 203
Validation binary_cross_entropy = 0.839130
Epoch 204
Loss = 4.8033e-03, PNorm = 70.8092, GNorm = 0.0348, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.852968
Epoch 205
Validation binary_cross_entropy = 0.867518
Epoch 206
Validation binary_cross_entropy = 0.890648
Epoch 207
Validation binary_cross_entropy = 0.917655
Epoch 208
Validation binary_cross_entropy = 0.964658
Epoch 209
Loss = 2.5401e-03, PNorm = 70.8261, GNorm = 0.1866, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.995126
Epoch 210
Validation binary_cross_entropy = 0.991009
Epoch 211
Validation binary_cross_entropy = 0.950403
Epoch 212
Validation binary_cross_entropy = 0.915955
Epoch 213
Validation binary_cross_entropy = 0.903549
Epoch 214
Loss = 1.9052e-03, PNorm = 70.8472, GNorm = 0.0974, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.903386
Epoch 215
Validation binary_cross_entropy = 0.961323
Epoch 216
Validation binary_cross_entropy = 1.010725
Epoch 217
Validation binary_cross_entropy = 1.027734
Epoch 218
Validation binary_cross_entropy = 0.982220
Epoch 219
Loss = 2.5973e-03, PNorm = 70.8799, GNorm = 0.0313, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.939324
Epoch 220
Validation binary_cross_entropy = 0.906338
Epoch 221
Validation binary_cross_entropy = 0.888740
Epoch 222
Validation binary_cross_entropy = 0.877627
Epoch 223
Validation binary_cross_entropy = 0.875888
Epoch 224
Loss = 1.9480e-03, PNorm = 70.8980, GNorm = 0.2678, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.878159
Epoch 225
Validation binary_cross_entropy = 0.883629
Epoch 226
Validation binary_cross_entropy = 0.912386
Epoch 227
Validation binary_cross_entropy = 0.951949
Epoch 228
Validation binary_cross_entropy = 0.985135
Epoch 229
Loss = 5.1911e-03, PNorm = 70.9096, GNorm = 0.6062, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.000240
Epoch 230
Validation binary_cross_entropy = 0.958698
Epoch 231
Validation binary_cross_entropy = 0.921418
Epoch 232
Validation binary_cross_entropy = 0.892414
Epoch 233
Validation binary_cross_entropy = 0.874145
Epoch 234
Loss = 4.3713e-02, PNorm = 70.9253, GNorm = 2.1848, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.875403
Epoch 235
Validation binary_cross_entropy = 0.903934
Epoch 236
Validation binary_cross_entropy = 0.935077
Epoch 237
Validation binary_cross_entropy = 0.959617
Epoch 238
Validation binary_cross_entropy = 0.975745
Epoch 239
Loss = 3.9345e-03, PNorm = 70.9556, GNorm = 0.0717, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.970390
Epoch 240
Validation binary_cross_entropy = 0.934461
Epoch 241
Validation binary_cross_entropy = 0.909591
Epoch 242
Validation binary_cross_entropy = 0.895614
Epoch 243
Validation binary_cross_entropy = 0.886692
Epoch 244
Loss = 8.8415e-03, PNorm = 70.9702, GNorm = 1.1403, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.893571
Epoch 245
Validation binary_cross_entropy = 0.913951
Epoch 246
Validation binary_cross_entropy = 0.934399
Epoch 247
Validation binary_cross_entropy = 0.950427
Epoch 248
Validation binary_cross_entropy = 0.961800
Epoch 249
Loss = 7.9423e-04, PNorm = 70.9833, GNorm = 0.0660, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.970559
Epoch 250
Validation binary_cross_entropy = 0.975391
Epoch 251
Validation binary_cross_entropy = 0.976885
Epoch 252
Validation binary_cross_entropy = 0.974825
Epoch 253
Validation binary_cross_entropy = 0.993582
Epoch 254
Loss = 8.7684e-04, PNorm = 70.9961, GNorm = 0.1106, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.033231
Epoch 255
Validation binary_cross_entropy = 1.047495
Epoch 256
Validation binary_cross_entropy = 0.969534
Epoch 257
Validation binary_cross_entropy = 0.914899
Epoch 258
Validation binary_cross_entropy = 0.895350
Epoch 259
Loss = 6.5412e-04, PNorm = 71.0164, GNorm = 0.0227, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.888788
Epoch 260
Validation binary_cross_entropy = 0.888112
Epoch 261
Validation binary_cross_entropy = 0.912455
Epoch 262
Validation binary_cross_entropy = 0.935840
Epoch 263
Validation binary_cross_entropy = 0.966867
Epoch 264
Loss = 2.3321e-02, PNorm = 71.0357, GNorm = 0.1981, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.052412
Epoch 265
Validation binary_cross_entropy = 1.090924
Epoch 266
Validation binary_cross_entropy = 1.052733
Epoch 267
Validation binary_cross_entropy = 0.940100
Epoch 268
Validation binary_cross_entropy = 0.871771
Epoch 269
Loss = 1.1719e-03, PNorm = 71.0628, GNorm = 0.0130, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.839508
Epoch 270
Validation binary_cross_entropy = 0.831880
Epoch 271
Validation binary_cross_entropy = 0.834302
Epoch 272
Validation binary_cross_entropy = 0.836756
Epoch 273
Validation binary_cross_entropy = 0.859898
Epoch 274
Loss = 3.5503e-04, PNorm = 71.1045, GNorm = 0.0071, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.897745
Epoch 275
Validation binary_cross_entropy = 0.972818
Epoch 276
Validation binary_cross_entropy = 1.036118
Epoch 277
Validation binary_cross_entropy = 1.068815
Epoch 278
Validation binary_cross_entropy = 1.013870
Epoch 279
Loss = 1.4405e-03, PNorm = 71.1410, GNorm = 0.1050, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.968400
Epoch 280
Validation binary_cross_entropy = 0.959986
Epoch 281
Validation binary_cross_entropy = 0.977095
Epoch 282
Validation binary_cross_entropy = 0.990224
Epoch 283
Validation binary_cross_entropy = 0.991458
Epoch 284
Loss = 3.6170e-02, PNorm = 71.1777, GNorm = 0.0232, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.930295
Epoch 285
Validation binary_cross_entropy = 0.898742
Epoch 286
Validation binary_cross_entropy = 0.881303
Epoch 287
Validation binary_cross_entropy = 0.881779
Epoch 288
Validation binary_cross_entropy = 0.895390
Epoch 289
Loss = 5.2969e-02, PNorm = 71.2251, GNorm = 0.0820, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.947154
Epoch 290
Validation binary_cross_entropy = 1.011398
Epoch 291
Validation binary_cross_entropy = 1.067133
Epoch 292
Validation binary_cross_entropy = 1.102844
Epoch 293
Validation binary_cross_entropy = 1.049909
Epoch 294
Loss = 2.6526e-03, PNorm = 71.2617, GNorm = 0.0233, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.990054
Epoch 295
Validation binary_cross_entropy = 0.945209
Epoch 296
Validation binary_cross_entropy = 0.912639
Epoch 297
Validation binary_cross_entropy = 0.893651
Epoch 298
Validation binary_cross_entropy = 0.885333
Epoch 299
Loss = 3.6572e-03, PNorm = 71.2816, GNorm = 0.5017, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.881924
Model 0 best validation binary_cross_entropy = 0.371793 on epoch 3
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.249590
Ensemble test binary_cross_entropy = 0.249590
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.166739
Epoch 1
Validation binary_cross_entropy = 0.392666
Epoch 2
Validation binary_cross_entropy = 1.760636
Epoch 3
Validation binary_cross_entropy = 1.191525
Epoch 4
Loss = 9.4687e-01, PNorm = 68.1473, GNorm = 7.9186, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.497224
Epoch 5
Validation binary_cross_entropy = 0.455379
Epoch 6
Validation binary_cross_entropy = 1.756435
Epoch 7
Validation binary_cross_entropy = 0.532457
Epoch 8
Validation binary_cross_entropy = 0.685758
Epoch 9
Loss = 6.5519e-01, PNorm = 68.3194, GNorm = 4.4787, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.802938
Epoch 10
Validation binary_cross_entropy = 0.861293
Epoch 11
Validation binary_cross_entropy = 0.536116
Epoch 12
Validation binary_cross_entropy = 0.595040
Epoch 13
Validation binary_cross_entropy = 0.718961
Epoch 14
Loss = 4.0439e-01, PNorm = 68.5058, GNorm = 9.4586, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.861671
Epoch 15
Validation binary_cross_entropy = 0.647558
Epoch 16
Validation binary_cross_entropy = 0.704099
Epoch 17
Validation binary_cross_entropy = 0.641662
Epoch 18
Validation binary_cross_entropy = 0.643325
Epoch 19
Loss = 2.5338e-01, PNorm = 68.6333, GNorm = 3.7954, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.580035
Epoch 20
Validation binary_cross_entropy = 0.571315
Epoch 21
Validation binary_cross_entropy = 0.562323
Epoch 22
Validation binary_cross_entropy = 0.563057
Epoch 23
Validation binary_cross_entropy = 0.543996
Epoch 24
Loss = 7.9279e-02, PNorm = 68.7178, GNorm = 1.4660, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.538776
Epoch 25
Validation binary_cross_entropy = 0.554983
Epoch 26
Validation binary_cross_entropy = 0.582731
Epoch 27
Validation binary_cross_entropy = 0.558190
Epoch 28
Validation binary_cross_entropy = 0.605377
Epoch 29
Loss = 1.7454e-01, PNorm = 68.7807, GNorm = 3.5227, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.603558
Epoch 30
Validation binary_cross_entropy = 0.621518
Epoch 31
Validation binary_cross_entropy = 0.630235
Epoch 32
Validation binary_cross_entropy = 0.632354
Epoch 33
Validation binary_cross_entropy = 0.646451
Epoch 34
Loss = 1.3382e-01, PNorm = 68.8473, GNorm = 3.9454, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.612761
Epoch 35
Validation binary_cross_entropy = 0.530733
Epoch 36
Validation binary_cross_entropy = 0.511487
Epoch 37
Validation binary_cross_entropy = 0.525321
Epoch 38
Validation binary_cross_entropy = 0.545014
Epoch 39
Loss = 6.7486e-02, PNorm = 68.9046, GNorm = 2.6623, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.524318
Epoch 40
Validation binary_cross_entropy = 0.538001
Epoch 41
Validation binary_cross_entropy = 0.558352
Epoch 42
Validation binary_cross_entropy = 0.586094
Epoch 43
Validation binary_cross_entropy = 0.645632
Epoch 44
Loss = 1.1262e-01, PNorm = 68.9624, GNorm = 2.5848, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.649532
Epoch 45
Validation binary_cross_entropy = 0.631881
Epoch 46
Validation binary_cross_entropy = 0.616936
Epoch 47
Validation binary_cross_entropy = 0.626982
Epoch 48
Validation binary_cross_entropy = 0.609485
Epoch 49
Loss = 5.5541e-02, PNorm = 69.0066, GNorm = 1.9643, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.568048
Epoch 50
Validation binary_cross_entropy = 0.579134
Epoch 51
Validation binary_cross_entropy = 0.661505
Epoch 52
Validation binary_cross_entropy = 0.655322
Epoch 53
Validation binary_cross_entropy = 0.628454
Epoch 54
Loss = 1.1232e-01, PNorm = 69.0644, GNorm = 3.6308, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.632302
Epoch 55
Validation binary_cross_entropy = 0.643187
Epoch 56
Validation binary_cross_entropy = 0.658024
Epoch 57
Validation binary_cross_entropy = 0.663367
Epoch 58
Validation binary_cross_entropy = 0.647489
Epoch 59
Loss = 8.3383e-02, PNorm = 69.1312, GNorm = 3.6299, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.633838
Epoch 60
Validation binary_cross_entropy = 0.624849
Epoch 61
Validation binary_cross_entropy = 0.622259
Epoch 62
Validation binary_cross_entropy = 0.636840
Epoch 63
Validation binary_cross_entropy = 0.654113
Epoch 64
Loss = 1.7696e-02, PNorm = 69.1827, GNorm = 0.3770, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.666516
Epoch 65
Validation binary_cross_entropy = 0.786735
Epoch 66
Validation binary_cross_entropy = 0.689365
Epoch 67
Validation binary_cross_entropy = 0.601125
Epoch 68
Validation binary_cross_entropy = 0.629585
Epoch 69
Loss = 8.2568e-02, PNorm = 69.2382, GNorm = 4.6261, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.618220
Epoch 70
Validation binary_cross_entropy = 0.665134
Epoch 71
Validation binary_cross_entropy = 0.703635
Epoch 72
Validation binary_cross_entropy = 0.620483
Epoch 73
Validation binary_cross_entropy = 0.674897
Epoch 74
Loss = 1.2258e-01, PNorm = 69.3086, GNorm = 4.0013, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.612251
Epoch 75
Validation binary_cross_entropy = 0.685484
Epoch 76
Validation binary_cross_entropy = 0.716209
Epoch 77
Validation binary_cross_entropy = 0.636497
Epoch 78
Validation binary_cross_entropy = 0.608667
Epoch 79
Loss = 1.3436e-02, PNorm = 69.3746, GNorm = 0.6500, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.620017
Epoch 80
Validation binary_cross_entropy = 0.620797
Epoch 81
Validation binary_cross_entropy = 0.616623
Epoch 82
Validation binary_cross_entropy = 0.638591
Epoch 83
Validation binary_cross_entropy = 0.690809
Epoch 84
Loss = 5.3316e-02, PNorm = 69.4441, GNorm = 0.9819, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.639675
Epoch 85
Validation binary_cross_entropy = 0.616094
Epoch 86
Validation binary_cross_entropy = 0.616736
Epoch 87
Validation binary_cross_entropy = 0.633696
Epoch 88
Validation binary_cross_entropy = 0.704609
Epoch 89
Loss = 3.9666e-02, PNorm = 69.5229, GNorm = 2.3981, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.752719
Epoch 90
Validation binary_cross_entropy = 0.713150
Epoch 91
Validation binary_cross_entropy = 0.666292
Epoch 92
Validation binary_cross_entropy = 0.678786
Epoch 93
Validation binary_cross_entropy = 0.688749
Epoch 94
Loss = 1.5268e-02, PNorm = 69.5694, GNorm = 0.4072, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.700784
Epoch 95
Validation binary_cross_entropy = 0.725348
Epoch 96
Validation binary_cross_entropy = 0.740346
Epoch 97
Validation binary_cross_entropy = 0.738441
Epoch 98
Validation binary_cross_entropy = 0.727352
Epoch 99
Loss = 1.6135e-02, PNorm = 69.5963, GNorm = 0.7785, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.705598
Epoch 100
Validation binary_cross_entropy = 0.699065
Epoch 101
Validation binary_cross_entropy = 0.708786
Epoch 102
Validation binary_cross_entropy = 0.715414
Epoch 103
Validation binary_cross_entropy = 0.705561
Epoch 104
Loss = 1.6848e-02, PNorm = 69.6309, GNorm = 1.1554, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.682247
Epoch 105
Validation binary_cross_entropy = 0.663592
Epoch 106
Validation binary_cross_entropy = 0.665698
Epoch 107
Validation binary_cross_entropy = 0.696757
Epoch 108
Validation binary_cross_entropy = 0.753952
Epoch 109
Loss = 1.3751e-02, PNorm = 69.6658, GNorm = 1.2538, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.790712
Epoch 110
Validation binary_cross_entropy = 0.776424
Epoch 111
Validation binary_cross_entropy = 0.735075
Epoch 112
Validation binary_cross_entropy = 0.708886
Epoch 113
Validation binary_cross_entropy = 0.695069
Epoch 114
Loss = 3.8858e-02, PNorm = 69.7072, GNorm = 1.4531, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.714605
Epoch 115
Validation binary_cross_entropy = 0.753151
Epoch 116
Validation binary_cross_entropy = 0.765957
Epoch 117
Validation binary_cross_entropy = 0.749641
Epoch 118
Validation binary_cross_entropy = 0.740093
Epoch 119
Loss = 4.6356e-03, PNorm = 69.7562, GNorm = 0.0890, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.733588
Epoch 120
Validation binary_cross_entropy = 0.704581
Epoch 121
Validation binary_cross_entropy = 0.700460
Epoch 122
Validation binary_cross_entropy = 0.707891
Epoch 123
Validation binary_cross_entropy = 0.714164
Epoch 124
Loss = 2.0486e-02, PNorm = 69.7947, GNorm = 0.8249, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.736279
Epoch 125
Validation binary_cross_entropy = 0.788820
Epoch 126
Validation binary_cross_entropy = 0.805037
Epoch 127
Validation binary_cross_entropy = 0.806301
Epoch 128
Validation binary_cross_entropy = 0.789121
Epoch 129
Loss = 4.9674e-03, PNorm = 69.8341, GNorm = 0.1167, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.765919
Epoch 130
Validation binary_cross_entropy = 0.750639
Epoch 131
Validation binary_cross_entropy = 0.743054
Epoch 132
Validation binary_cross_entropy = 0.754910
Epoch 133
Validation binary_cross_entropy = 0.788210
Epoch 134
Loss = 9.9244e-03, PNorm = 69.8613, GNorm = 0.3882, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.799673
Epoch 135
Validation binary_cross_entropy = 0.790390
Epoch 136
Validation binary_cross_entropy = 0.733213
Epoch 137
Validation binary_cross_entropy = 0.717861
Epoch 138
Validation binary_cross_entropy = 0.722942
Epoch 139
Loss = 5.4218e-03, PNorm = 69.9020, GNorm = 0.0905, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.735247
Epoch 140
Validation binary_cross_entropy = 0.756390
Epoch 141
Validation binary_cross_entropy = 0.780869
Epoch 142
Validation binary_cross_entropy = 0.792517
Epoch 143
Validation binary_cross_entropy = 0.770039
Epoch 144
Loss = 4.8984e-02, PNorm = 69.9355, GNorm = 0.0631, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.783321
Epoch 145
Validation binary_cross_entropy = 0.798261
Epoch 146
Validation binary_cross_entropy = 0.762607
Epoch 147
Validation binary_cross_entropy = 0.716574
Epoch 148
Validation binary_cross_entropy = 0.712003
Epoch 149
Loss = 2.3642e-02, PNorm = 69.9776, GNorm = 1.9986, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.732683
Epoch 150
Validation binary_cross_entropy = 0.806482
Epoch 151
Validation binary_cross_entropy = 0.888118
Epoch 152
Validation binary_cross_entropy = 0.893937
Epoch 153
Validation binary_cross_entropy = 0.842262
Epoch 154
Loss = 9.7276e-03, PNorm = 70.0248, GNorm = 0.5991, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.783496
Epoch 155
Validation binary_cross_entropy = 0.743638
Epoch 156
Validation binary_cross_entropy = 0.729807
Epoch 157
Validation binary_cross_entropy = 0.735324
Epoch 158
Validation binary_cross_entropy = 0.768193
Epoch 159
Loss = 3.0651e-02, PNorm = 70.0514, GNorm = 3.3007, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.822041
Epoch 160
Validation binary_cross_entropy = 0.895501
Epoch 161
Validation binary_cross_entropy = 0.889342
Epoch 162
Validation binary_cross_entropy = 0.804507
Epoch 163
Validation binary_cross_entropy = 0.739176
Epoch 164
Loss = 1.7434e-02, PNorm = 70.1143, GNorm = 1.0934, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.738846
Epoch 165
Validation binary_cross_entropy = 0.738385
Epoch 166
Validation binary_cross_entropy = 0.765970
Epoch 167
Validation binary_cross_entropy = 0.814828
Epoch 168
Validation binary_cross_entropy = 0.844627
Epoch 169
Loss = 4.3397e-03, PNorm = 70.1541, GNorm = 0.3166, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.868393
Epoch 170
Validation binary_cross_entropy = 0.861855
Epoch 171
Validation binary_cross_entropy = 0.834338
Epoch 172
Validation binary_cross_entropy = 0.817351
Epoch 173
Validation binary_cross_entropy = 0.810411
Epoch 174
Loss = 2.2054e-03, PNorm = 70.2048, GNorm = 0.0565, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.810045
Epoch 175
Validation binary_cross_entropy = 0.815604
Epoch 176
Validation binary_cross_entropy = 0.846044
Epoch 177
Validation binary_cross_entropy = 0.901862
Epoch 178
Validation binary_cross_entropy = 0.989532
Epoch 179
Loss = 2.9664e-02, PNorm = 70.2339, GNorm = 2.1322, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.002941
Epoch 180
Validation binary_cross_entropy = 0.846553
Epoch 181
Validation binary_cross_entropy = 0.795361
Epoch 182
Validation binary_cross_entropy = 0.802390
Epoch 183
Validation binary_cross_entropy = 0.768879
Epoch 184
Loss = 6.5979e-02, PNorm = 70.2763, GNorm = 3.2420, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.816476
Epoch 185
Validation binary_cross_entropy = 0.957058
Epoch 186
Validation binary_cross_entropy = 0.861316
Epoch 187
Validation binary_cross_entropy = 0.740882
Epoch 188
Validation binary_cross_entropy = 0.829700
Epoch 189
Loss = 6.9288e-02, PNorm = 70.3325, GNorm = 4.1175, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.822200
Epoch 190
Validation binary_cross_entropy = 0.750772
Epoch 191
Validation binary_cross_entropy = 0.861646
Epoch 192
Validation binary_cross_entropy = 0.938844
Epoch 193
Validation binary_cross_entropy = 0.935632
Epoch 194
Loss = 1.8368e-02, PNorm = 70.4215, GNorm = 0.8901, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.874137
Epoch 195
Validation binary_cross_entropy = 0.819775
Epoch 196
Validation binary_cross_entropy = 0.790833
Epoch 197
Validation binary_cross_entropy = 0.771146
Epoch 198
Validation binary_cross_entropy = 0.763714
Epoch 199
Loss = 1.0174e-02, PNorm = 70.5412, GNorm = 1.0655, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.770146
Epoch 200
Validation binary_cross_entropy = 0.781812
Epoch 201
Validation binary_cross_entropy = 0.814412
Epoch 202
Validation binary_cross_entropy = 0.847469
Epoch 203
Validation binary_cross_entropy = 0.861318
Epoch 204
Loss = 2.0181e-02, PNorm = 70.6189, GNorm = 1.7301, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.889955
Epoch 205
Validation binary_cross_entropy = 0.931262
Epoch 206
Validation binary_cross_entropy = 0.950274
Epoch 207
Validation binary_cross_entropy = 0.906428
Epoch 208
Validation binary_cross_entropy = 0.863219
Epoch 209
Loss = 9.7033e-03, PNorm = 70.6772, GNorm = 0.0821, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.861442
Epoch 210
Validation binary_cross_entropy = 0.877288
Epoch 211
Validation binary_cross_entropy = 0.901073
Epoch 212
Validation binary_cross_entropy = 0.902856
Epoch 213
Validation binary_cross_entropy = 0.901882
Epoch 214
Loss = 4.7570e-03, PNorm = 70.7627, GNorm = 0.3733, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.900586
Epoch 215
Validation binary_cross_entropy = 0.903068
Epoch 216
Validation binary_cross_entropy = 0.908975
Epoch 217
Validation binary_cross_entropy = 0.909237
Epoch 218
Validation binary_cross_entropy = 0.904286
Epoch 219
Loss = 7.3549e-03, PNorm = 70.8114, GNorm = 0.6536, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.903538
Epoch 220
Validation binary_cross_entropy = 0.903881
Epoch 221
Validation binary_cross_entropy = 0.905124
Epoch 222
Validation binary_cross_entropy = 0.904109
Epoch 223
Validation binary_cross_entropy = 0.903549
Epoch 224
Loss = 3.0151e-03, PNorm = 70.8396, GNorm = 0.3084, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.905654
Epoch 225
Validation binary_cross_entropy = 0.905980
Epoch 226
Validation binary_cross_entropy = 0.897432
Epoch 227
Validation binary_cross_entropy = 0.889590
Epoch 228
Validation binary_cross_entropy = 0.896930
Epoch 229
Loss = 6.3443e-03, PNorm = 70.8662, GNorm = 0.7992, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.909172
Epoch 230
Validation binary_cross_entropy = 0.908072
Epoch 231
Validation binary_cross_entropy = 0.909205
Epoch 232
Validation binary_cross_entropy = 0.907719
Epoch 233
Validation binary_cross_entropy = 0.902048
Epoch 234
Loss = 4.4263e-03, PNorm = 70.8769, GNorm = 0.5530, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.900895
Epoch 235
Validation binary_cross_entropy = 0.916355
Epoch 236
Validation binary_cross_entropy = 0.908696
Epoch 237
Validation binary_cross_entropy = 0.893031
Epoch 238
Validation binary_cross_entropy = 0.878602
Epoch 239
Loss = 5.6750e-03, PNorm = 70.8927, GNorm = 0.9920, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.872460
Epoch 240
Validation binary_cross_entropy = 0.916837
Epoch 241
Validation binary_cross_entropy = 0.985894
Epoch 242
Validation binary_cross_entropy = 0.963931
Epoch 243
Validation binary_cross_entropy = 0.897854
Epoch 244
Loss = 8.4180e-03, PNorm = 70.9277, GNorm = 1.4046, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.862211
Epoch 245
Validation binary_cross_entropy = 0.851390
Epoch 246
Validation binary_cross_entropy = 0.848148
Epoch 247
Validation binary_cross_entropy = 0.861511
Epoch 248
Validation binary_cross_entropy = 0.900615
Epoch 249
Loss = 8.9723e-03, PNorm = 70.9645, GNorm = 0.0272, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.935884
Epoch 250
Validation binary_cross_entropy = 0.964711
Epoch 251
Validation binary_cross_entropy = 0.971525
Epoch 252
Validation binary_cross_entropy = 0.959132
Epoch 253
Validation binary_cross_entropy = 0.949087
Epoch 254
Loss = 3.0529e-03, PNorm = 70.9960, GNorm = 0.0938, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.936559
Epoch 255
Validation binary_cross_entropy = 0.923452
Epoch 256
Validation binary_cross_entropy = 0.911755
Epoch 257
Validation binary_cross_entropy = 0.900095
Epoch 258
Validation binary_cross_entropy = 0.893912
Epoch 259
Loss = 2.1527e-02, PNorm = 71.0229, GNorm = 3.5216, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.915799
Epoch 260
Validation binary_cross_entropy = 0.957247
Epoch 261
Validation binary_cross_entropy = 0.964681
Epoch 262
Validation binary_cross_entropy = 0.947366
Epoch 263
Validation binary_cross_entropy = 0.930396
Epoch 264
Loss = 2.3351e-03, PNorm = 71.0451, GNorm = 0.1428, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.915635
Epoch 265
Validation binary_cross_entropy = 0.901244
Epoch 266
Validation binary_cross_entropy = 0.888733
Epoch 267
Validation binary_cross_entropy = 0.879953
Epoch 268
Validation binary_cross_entropy = 0.873445
Epoch 269
Loss = 3.0247e-03, PNorm = 71.0654, GNorm = 0.2509, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.874001
Epoch 270
Validation binary_cross_entropy = 0.880869
Epoch 271
Validation binary_cross_entropy = 0.907177
Epoch 272
Validation binary_cross_entropy = 0.933334
Epoch 273
Validation binary_cross_entropy = 0.955458
Epoch 274
Loss = 2.2104e-03, PNorm = 71.0784, GNorm = 0.0455, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.970653
Epoch 275
Validation binary_cross_entropy = 0.943522
Epoch 276
Validation binary_cross_entropy = 0.924673
Epoch 277
Validation binary_cross_entropy = 0.911995
Epoch 278
Validation binary_cross_entropy = 0.904055
Epoch 279
Loss = 9.5878e-04, PNorm = 71.0924, GNorm = 0.0491, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.898979
Epoch 280
Validation binary_cross_entropy = 0.898910
Epoch 281
Validation binary_cross_entropy = 0.909451
Epoch 282
Validation binary_cross_entropy = 0.923814
Epoch 283
Validation binary_cross_entropy = 0.936967
Epoch 284
Loss = 3.9472e-03, PNorm = 71.1028, GNorm = 0.0733, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.956335
Epoch 285
Validation binary_cross_entropy = 0.966633
Epoch 286
Validation binary_cross_entropy = 0.967584
Epoch 287
Validation binary_cross_entropy = 0.971108
Epoch 288
Validation binary_cross_entropy = 0.976904
Epoch 289
Loss = 1.9804e-03, PNorm = 71.1116, GNorm = 0.0934, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.973786
Epoch 290
Validation binary_cross_entropy = 0.967068
Epoch 291
Validation binary_cross_entropy = 0.958097
Epoch 292
Validation binary_cross_entropy = 0.943943
Epoch 293
Validation binary_cross_entropy = 0.930013
Epoch 294
Loss = 1.6692e-03, PNorm = 71.1256, GNorm = 0.0403, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.922419
Epoch 295
Validation binary_cross_entropy = 0.916963
Epoch 296
Validation binary_cross_entropy = 0.913859
Epoch 297
Validation binary_cross_entropy = 0.913283
Epoch 298
Validation binary_cross_entropy = 0.913747
Epoch 299
Loss = 3.2855e-03, PNorm = 71.1383, GNorm = 0.1470, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.920184
Model 0 best validation binary_cross_entropy = 0.392666 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.216313
Ensemble test binary_cross_entropy = 0.216313
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.547399
Epoch 1
Validation binary_cross_entropy = 0.365022
Epoch 2
Validation binary_cross_entropy = 1.143579
Epoch 3
Validation binary_cross_entropy = 0.446647
Epoch 4
Loss = 7.8650e-01, PNorm = 68.1481, GNorm = 15.6945, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.426408
Epoch 5
Validation binary_cross_entropy = 1.097843
Epoch 6
Validation binary_cross_entropy = 0.614865
Epoch 7
Validation binary_cross_entropy = 1.148377
Epoch 8
Validation binary_cross_entropy = 1.055923
Epoch 9
Loss = 4.8573e-01, PNorm = 68.3080, GNorm = 8.2566, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.699122
Epoch 10
Validation binary_cross_entropy = 0.665465
Epoch 11
Validation binary_cross_entropy = 1.081926
Epoch 12
Validation binary_cross_entropy = 0.732438
Epoch 13
Validation binary_cross_entropy = 0.739702
Epoch 14
Loss = 3.8522e-01, PNorm = 68.4877, GNorm = 6.8146, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.809071
Epoch 15
Validation binary_cross_entropy = 1.012837
Epoch 16
Validation binary_cross_entropy = 0.676377
Epoch 17
Validation binary_cross_entropy = 0.629515
Epoch 18
Validation binary_cross_entropy = 0.677349
Epoch 19
Loss = 1.2662e-01, PNorm = 68.6182, GNorm = 3.8214, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.923493
Epoch 20
Validation binary_cross_entropy = 0.543204
Epoch 21
Validation binary_cross_entropy = 0.564364
Epoch 22
Validation binary_cross_entropy = 0.512097
Epoch 23
Validation binary_cross_entropy = 0.587846
Epoch 24
Loss = 1.5280e-01, PNorm = 68.7137, GNorm = 3.3571, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.569772
Epoch 25
Validation binary_cross_entropy = 0.629151
Epoch 26
Validation binary_cross_entropy = 0.578418
Epoch 27
Validation binary_cross_entropy = 0.576444
Epoch 28
Validation binary_cross_entropy = 0.558629
Epoch 29
Loss = 7.5203e-02, PNorm = 68.7835, GNorm = 2.0103, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.573524
Epoch 30
Validation binary_cross_entropy = 0.591726
Epoch 31
Validation binary_cross_entropy = 0.603795
Epoch 32
Validation binary_cross_entropy = 0.661220
Epoch 33
Validation binary_cross_entropy = 0.640153
Epoch 34
Loss = 2.1720e-01, PNorm = 68.8481, GNorm = 2.8093, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.676812
Epoch 35
Validation binary_cross_entropy = 0.633066
Epoch 36
Validation binary_cross_entropy = 0.586910
Epoch 37
Validation binary_cross_entropy = 0.721704
Epoch 38
Validation binary_cross_entropy = 0.548867
Epoch 39
Loss = 1.6532e-01, PNorm = 68.9182, GNorm = 6.6571, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.597419
Epoch 40
Validation binary_cross_entropy = 0.528908
Epoch 41
Validation binary_cross_entropy = 0.596912
Epoch 42
Validation binary_cross_entropy = 0.667221
Epoch 43
Validation binary_cross_entropy = 0.621494
Epoch 44
Loss = 9.2978e-02, PNorm = 69.0002, GNorm = 1.8624, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.709793
Epoch 45
Validation binary_cross_entropy = 0.642865
Epoch 46
Validation binary_cross_entropy = 0.664255
Epoch 47
Validation binary_cross_entropy = 0.666113
Epoch 48
Validation binary_cross_entropy = 0.652442
Epoch 49
Loss = 9.7856e-02, PNorm = 69.1010, GNorm = 1.3056, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.664905
Epoch 50
Validation binary_cross_entropy = 0.646776
Epoch 51
Validation binary_cross_entropy = 0.621638
Epoch 52
Validation binary_cross_entropy = 0.657072
Epoch 53
Validation binary_cross_entropy = 0.678371
Epoch 54
Loss = 9.2255e-02, PNorm = 69.1774, GNorm = 3.8870, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.681064
Epoch 55
Validation binary_cross_entropy = 0.669551
Epoch 56
Validation binary_cross_entropy = 0.636438
Epoch 57
Validation binary_cross_entropy = 0.633203
Epoch 58
Validation binary_cross_entropy = 0.636135
Epoch 59
Loss = 5.9882e-02, PNorm = 69.2570, GNorm = 0.5959, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.634095
Epoch 60
Validation binary_cross_entropy = 0.666139
Epoch 61
Validation binary_cross_entropy = 0.661147
Epoch 62
Validation binary_cross_entropy = 0.628593
Epoch 63
Validation binary_cross_entropy = 0.612228
Epoch 64
Loss = 6.4065e-02, PNorm = 69.3536, GNorm = 0.7593, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.581515
Epoch 65
Validation binary_cross_entropy = 0.584707
Epoch 66
Validation binary_cross_entropy = 0.633526
Epoch 67
Validation binary_cross_entropy = 0.620996
Epoch 68
Validation binary_cross_entropy = 0.592811
Epoch 69
Loss = 2.7818e-02, PNorm = 69.4285, GNorm = 0.3421, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.608173
Epoch 70
Validation binary_cross_entropy = 0.620124
Epoch 71
Validation binary_cross_entropy = 0.668871
Epoch 72
Validation binary_cross_entropy = 0.704300
Epoch 73
Validation binary_cross_entropy = 0.676283
Epoch 74
Loss = 1.2914e-02, PNorm = 69.4671, GNorm = 0.3837, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.632860
Epoch 75
Validation binary_cross_entropy = 0.627647
Epoch 76
Validation binary_cross_entropy = 0.644610
Epoch 77
Validation binary_cross_entropy = 0.697515
Epoch 78
Validation binary_cross_entropy = 0.738513
Epoch 79
Loss = 3.3288e-02, PNorm = 69.4972, GNorm = 1.3566, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.707619
Epoch 80
Validation binary_cross_entropy = 0.640725
Epoch 81
Validation binary_cross_entropy = 0.604782
Epoch 82
Validation binary_cross_entropy = 0.609456
Epoch 83
Validation binary_cross_entropy = 0.620898
Epoch 84
Loss = 9.4487e-03, PNorm = 69.5223, GNorm = 0.5277, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.629452
Epoch 85
Validation binary_cross_entropy = 0.635523
Epoch 86
Validation binary_cross_entropy = 0.634768
Epoch 87
Validation binary_cross_entropy = 0.656344
Epoch 88
Validation binary_cross_entropy = 0.674114
Epoch 89
Loss = 2.7364e-02, PNorm = 69.5479, GNorm = 0.5686, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.713162
Epoch 90
Validation binary_cross_entropy = 0.701940
Epoch 91
Validation binary_cross_entropy = 0.688047
Epoch 92
Validation binary_cross_entropy = 0.668956
Epoch 93
Validation binary_cross_entropy = 0.655424
Epoch 94
Loss = 4.0666e-02, PNorm = 69.5725, GNorm = 0.9796, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.666247
Epoch 95
Validation binary_cross_entropy = 0.741925
Epoch 96
Validation binary_cross_entropy = 0.789134
Epoch 97
Validation binary_cross_entropy = 0.679515
Epoch 98
Validation binary_cross_entropy = 0.627568
Epoch 99
Loss = 1.7494e-02, PNorm = 69.6229, GNorm = 1.3343, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.615295
Epoch 100
Validation binary_cross_entropy = 0.642715
Epoch 101
Validation binary_cross_entropy = 0.724209
Epoch 102
Validation binary_cross_entropy = 0.715663
Epoch 103
Validation binary_cross_entropy = 0.681052
Epoch 104
Loss = 3.4397e-02, PNorm = 69.6865, GNorm = 2.2051, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.667536
Epoch 105
Validation binary_cross_entropy = 0.682168
Epoch 106
Validation binary_cross_entropy = 0.796759
Epoch 107
Validation binary_cross_entropy = 0.849897
Epoch 108
Validation binary_cross_entropy = 0.698009
Epoch 109
Loss = 2.3255e-02, PNorm = 69.7573, GNorm = 0.6183, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.719786
Epoch 110
Validation binary_cross_entropy = 0.907056
Epoch 111
Validation binary_cross_entropy = 0.767600
Epoch 112
Validation binary_cross_entropy = 0.793434
Epoch 113
Validation binary_cross_entropy = 0.855216
Epoch 114
Loss = 6.6878e-02, PNorm = 69.8331, GNorm = 1.8885, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.778600
Epoch 115
Validation binary_cross_entropy = 0.709447
Epoch 116
Validation binary_cross_entropy = 0.697037
Epoch 117
Validation binary_cross_entropy = 0.716541
Epoch 118
Validation binary_cross_entropy = 0.754505
Epoch 119
Loss = 2.8141e-02, PNorm = 69.8894, GNorm = 1.6332, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.796678
Epoch 120
Validation binary_cross_entropy = 0.801908
Epoch 121
Validation binary_cross_entropy = 0.740825
Epoch 122
Validation binary_cross_entropy = 0.711398
Epoch 123
Validation binary_cross_entropy = 0.722324
Epoch 124
Loss = 6.1014e-02, PNorm = 69.9360, GNorm = 3.7374, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.784609
Epoch 125
Validation binary_cross_entropy = 0.782765
Epoch 126
Validation binary_cross_entropy = 0.723296
Epoch 127
Validation binary_cross_entropy = 0.685246
Epoch 128
Validation binary_cross_entropy = 0.694132
Epoch 129
Loss = 4.8363e-02, PNorm = 69.9802, GNorm = 1.8813, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.704832
Epoch 130
Validation binary_cross_entropy = 0.722638
Epoch 131
Validation binary_cross_entropy = 0.684637
Epoch 132
Validation binary_cross_entropy = 0.669294
Epoch 133
Validation binary_cross_entropy = 0.667046
Epoch 134
Loss = 6.2793e-03, PNorm = 70.0248, GNorm = 0.0839, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.681910
Epoch 135
Validation binary_cross_entropy = 0.758819
Epoch 136
Validation binary_cross_entropy = 0.791211
Epoch 137
Validation binary_cross_entropy = 0.751737
Epoch 138
Validation binary_cross_entropy = 0.705969
Epoch 139
Loss = 2.2243e-02, PNorm = 70.0586, GNorm = 0.5879, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.676215
Epoch 140
Validation binary_cross_entropy = 0.673617
Epoch 141
Validation binary_cross_entropy = 0.680563
Epoch 142
Validation binary_cross_entropy = 0.717142
Epoch 143
Validation binary_cross_entropy = 0.757715
Epoch 144
Loss = 1.3263e-02, PNorm = 70.0902, GNorm = 0.1658, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.762453
Epoch 145
Validation binary_cross_entropy = 0.731446
Epoch 146
Validation binary_cross_entropy = 0.668293
Epoch 147
Validation binary_cross_entropy = 0.680544
Epoch 148
Validation binary_cross_entropy = 0.696917
Epoch 149
Loss = 2.4839e-02, PNorm = 70.1222, GNorm = 1.8024, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.711499
Epoch 150
Validation binary_cross_entropy = 0.782921
Epoch 151
Validation binary_cross_entropy = 0.765449
Epoch 152
Validation binary_cross_entropy = 0.791429
Epoch 153
Validation binary_cross_entropy = 0.721244
Epoch 154
Loss = 4.2422e-02, PNorm = 70.1597, GNorm = 0.2337, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.604412
Epoch 155
Validation binary_cross_entropy = 0.609192
Epoch 156
Validation binary_cross_entropy = 0.663327
Epoch 157
Validation binary_cross_entropy = 0.744412
Epoch 158
Validation binary_cross_entropy = 0.791736
Epoch 159
Loss = 3.5384e-02, PNorm = 70.2098, GNorm = 3.1619, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.757617
Epoch 160
Validation binary_cross_entropy = 0.687356
Epoch 161
Validation binary_cross_entropy = 0.648346
Epoch 162
Validation binary_cross_entropy = 0.636536
Epoch 163
Validation binary_cross_entropy = 0.637166
Epoch 164
Loss = 1.2441e-02, PNorm = 70.2464, GNorm = 0.4029, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.647394
Epoch 165
Validation binary_cross_entropy = 0.682951
Epoch 166
Validation binary_cross_entropy = 0.739923
Epoch 167
Validation binary_cross_entropy = 0.760831
Epoch 168
Validation binary_cross_entropy = 0.739884
Epoch 169
Loss = 3.5290e-03, PNorm = 70.2791, GNorm = 0.0589, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.706420
Epoch 170
Validation binary_cross_entropy = 0.691696
Epoch 171
Validation binary_cross_entropy = 0.689939
Epoch 172
Validation binary_cross_entropy = 0.716126
Epoch 173
Validation binary_cross_entropy = 0.738858
Epoch 174
Loss = 8.8142e-03, PNorm = 70.3119, GNorm = 0.7130, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.753776
Epoch 175
Validation binary_cross_entropy = 0.754538
Epoch 176
Validation binary_cross_entropy = 0.761238
Epoch 177
Validation binary_cross_entropy = 0.770083
Epoch 178
Validation binary_cross_entropy = 0.758683
Epoch 179
Loss = 4.5728e-03, PNorm = 70.3647, GNorm = 0.1508, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.743687
Epoch 180
Validation binary_cross_entropy = 0.760759
Epoch 181
Validation binary_cross_entropy = 0.771453
Epoch 182
Validation binary_cross_entropy = 0.777648
Epoch 183
Validation binary_cross_entropy = 0.749850
Epoch 184
Loss = 1.1302e-02, PNorm = 70.4022, GNorm = 2.5087, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.744810
Epoch 185
Validation binary_cross_entropy = 0.754560
Epoch 186
Validation binary_cross_entropy = 0.763752
Epoch 187
Validation binary_cross_entropy = 0.768858
Epoch 188
Validation binary_cross_entropy = 0.761146
Epoch 189
Loss = 1.5299e-03, PNorm = 70.4316, GNorm = 0.0371, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.747028
Epoch 190
Validation binary_cross_entropy = 0.745307
Epoch 191
Validation binary_cross_entropy = 0.758220
Epoch 192
Validation binary_cross_entropy = 0.798091
Epoch 193
Validation binary_cross_entropy = 0.801241
Epoch 194
Loss = 2.0326e-03, PNorm = 70.4634, GNorm = 0.0882, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.802371
Epoch 195
Validation binary_cross_entropy = 0.803259
Epoch 196
Validation binary_cross_entropy = 0.805892
Epoch 197
Validation binary_cross_entropy = 0.795125
Epoch 198
Validation binary_cross_entropy = 0.775762
Epoch 199
Loss = 1.8678e-03, PNorm = 70.4966, GNorm = 0.1116, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.761023
Epoch 200
Validation binary_cross_entropy = 0.769385
Epoch 201
Validation binary_cross_entropy = 0.804495
Epoch 202
Validation binary_cross_entropy = 0.830695
Epoch 203
Validation binary_cross_entropy = 0.830787
Epoch 204
Loss = 1.0642e-02, PNorm = 70.5269, GNorm = 0.1409, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.800910
Epoch 205
Validation binary_cross_entropy = 0.779921
Epoch 206
Validation binary_cross_entropy = 0.770338
Epoch 207
Validation binary_cross_entropy = 0.798828
Epoch 208
Validation binary_cross_entropy = 0.831727
Epoch 209
Loss = 6.3819e-03, PNorm = 70.5533, GNorm = 0.0959, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.858612
Epoch 210
Validation binary_cross_entropy = 0.877235
Epoch 211
Validation binary_cross_entropy = 0.891082
Epoch 212
Validation binary_cross_entropy = 0.889049
Epoch 213
Validation binary_cross_entropy = 0.875637
Epoch 214
Loss = 8.2794e-03, PNorm = 70.5741, GNorm = 1.2399, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.867607
Epoch 215
Validation binary_cross_entropy = 0.870945
Epoch 216
Validation binary_cross_entropy = 0.872652
Epoch 217
Validation binary_cross_entropy = 0.878898
Epoch 218
Validation binary_cross_entropy = 0.882451
Epoch 219
Loss = 4.0073e-03, PNorm = 70.6137, GNorm = 0.1690, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.886817
Epoch 220
Validation binary_cross_entropy = 0.901122
Epoch 221
Validation binary_cross_entropy = 0.914341
Epoch 222
Validation binary_cross_entropy = 0.920111
Epoch 223
Validation binary_cross_entropy = 0.912138
Epoch 224
Loss = 8.2251e-03, PNorm = 70.6465, GNorm = 0.0261, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.885234
Epoch 225
Validation binary_cross_entropy = 0.861396
Epoch 226
Validation binary_cross_entropy = 0.840636
Epoch 227
Validation binary_cross_entropy = 0.825982
Epoch 228
Validation binary_cross_entropy = 0.819956
Epoch 229
Loss = 9.1267e-04, PNorm = 70.6652, GNorm = 0.0649, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.820037
Epoch 230
Validation binary_cross_entropy = 0.825463
Epoch 231
Validation binary_cross_entropy = 0.834964
Epoch 232
Validation binary_cross_entropy = 0.846528
Epoch 233
Validation binary_cross_entropy = 0.859379
Epoch 234
Loss = 6.4050e-03, PNorm = 70.6829, GNorm = 0.0356, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.845962
Epoch 235
Validation binary_cross_entropy = 0.837598
Epoch 236
Validation binary_cross_entropy = 0.832902
Epoch 237
Validation binary_cross_entropy = 0.831559
Epoch 238
Validation binary_cross_entropy = 0.839097
Epoch 239
Loss = 3.1552e-03, PNorm = 70.7050, GNorm = 0.3067, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.852413
Epoch 240
Validation binary_cross_entropy = 0.863741
Epoch 241
Validation binary_cross_entropy = 0.866367
Epoch 242
Validation binary_cross_entropy = 0.864615
Epoch 243
Validation binary_cross_entropy = 0.861814
Epoch 244
Loss = 1.2067e-03, PNorm = 70.7251, GNorm = 0.0250, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.856282
Epoch 245
Validation binary_cross_entropy = 0.854536
Epoch 246
Validation binary_cross_entropy = 0.857193
Epoch 247
Validation binary_cross_entropy = 0.861471
Epoch 248
Validation binary_cross_entropy = 0.862636
Epoch 249
Loss = 1.0152e-03, PNorm = 70.7456, GNorm = 0.0480, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.862711
Epoch 250
Validation binary_cross_entropy = 0.861879
Epoch 251
Validation binary_cross_entropy = 0.857464
Epoch 252
Validation binary_cross_entropy = 0.855515
Epoch 253
Validation binary_cross_entropy = 0.855197
Epoch 254
Loss = 1.0315e-03, PNorm = 70.7673, GNorm = 0.0448, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.852360
Epoch 255
Validation binary_cross_entropy = 0.851710
Epoch 256
Validation binary_cross_entropy = 0.852388
Epoch 257
Validation binary_cross_entropy = 0.853596
Epoch 258
Validation binary_cross_entropy = 0.879945
Epoch 259
Loss = 1.6531e-03, PNorm = 70.8016, GNorm = 0.0788, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.925386
Epoch 260
Validation binary_cross_entropy = 0.939920
Epoch 261
Validation binary_cross_entropy = 0.923932
Epoch 262
Validation binary_cross_entropy = 0.899876
Epoch 263
Validation binary_cross_entropy = 0.879068
Epoch 264
Loss = 2.2420e-03, PNorm = 70.8438, GNorm = 0.0651, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.873431
Epoch 265
Validation binary_cross_entropy = 0.850221
Epoch 266
Validation binary_cross_entropy = 0.822029
Epoch 267
Validation binary_cross_entropy = 0.815811
Epoch 268
Validation binary_cross_entropy = 0.826615
Epoch 269
Loss = 1.0234e-03, PNorm = 70.8712, GNorm = 0.0986, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.838034
Epoch 270
Validation binary_cross_entropy = 0.846131
Epoch 271
Validation binary_cross_entropy = 0.850855
Epoch 272
Validation binary_cross_entropy = 0.854380
Epoch 273
Validation binary_cross_entropy = 0.845347
Epoch 274
Loss = 6.2059e-04, PNorm = 70.9025, GNorm = 0.0199, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.838920
Epoch 275
Validation binary_cross_entropy = 0.833302
Epoch 276
Validation binary_cross_entropy = 0.827870
Epoch 277
Validation binary_cross_entropy = 0.823811
Epoch 278
Validation binary_cross_entropy = 0.823353
Epoch 279
Loss = 1.1370e-02, PNorm = 70.9365, GNorm = 0.1381, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.853829
Epoch 280
Validation binary_cross_entropy = 0.884693
Epoch 281
Validation binary_cross_entropy = 0.910155
Epoch 282
Validation binary_cross_entropy = 0.906965
Epoch 283
Validation binary_cross_entropy = 0.880774
Epoch 284
Loss = 7.3408e-04, PNorm = 70.9859, GNorm = 0.0591, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.860105
Epoch 285
Validation binary_cross_entropy = 0.843526
Epoch 286
Validation binary_cross_entropy = 0.835955
Epoch 287
Validation binary_cross_entropy = 0.831935
Epoch 288
Validation binary_cross_entropy = 0.830524
Epoch 289
Loss = 1.1711e-03, PNorm = 71.0227, GNorm = 0.0287, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.832971
Epoch 290
Validation binary_cross_entropy = 0.859624
Epoch 291
Validation binary_cross_entropy = 0.885799
Epoch 292
Validation binary_cross_entropy = 0.908158
Epoch 293
Validation binary_cross_entropy = 0.928929
Epoch 294
Loss = 1.8669e-03, PNorm = 71.0593, GNorm = 0.0591, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.937918
Epoch 295
Validation binary_cross_entropy = 0.939239
Epoch 296
Validation binary_cross_entropy = 0.920246
Epoch 297
Validation binary_cross_entropy = 0.888020
Epoch 298
Validation binary_cross_entropy = 0.857020
Epoch 299
Loss = 4.7028e-04, PNorm = 71.0885, GNorm = 0.0275, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.834645
Model 0 best validation binary_cross_entropy = 0.365022 on epoch 1
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.208128
Ensemble test binary_cross_entropy = 0.208128
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.1, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1300, bias=False)
        (W_h): Linear(in_features=1300, out_features=1300, bias=False)
        (W_o): Linear(in_features=1433, out_features=1300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2500, out_features=1300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1300, out_features=1, bias=True)
  )
)
Number of parameters = 6,997,901
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.529347
Epoch 1
Validation binary_cross_entropy = 0.675400
Epoch 2
Validation binary_cross_entropy = 0.839111
Epoch 3
Validation binary_cross_entropy = 0.571774
Epoch 4
Loss = 6.6957e-01, PNorm = 68.1550, GNorm = 8.6283, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.783474
Epoch 5
Validation binary_cross_entropy = 0.728509
Epoch 6
Validation binary_cross_entropy = 1.137524
Epoch 7
Validation binary_cross_entropy = 0.692722
Epoch 8
Validation binary_cross_entropy = 0.780123
Epoch 9
Loss = 2.6337e-01, PNorm = 68.3226, GNorm = 2.9323, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.571532
Epoch 10
Validation binary_cross_entropy = 0.545064
Epoch 11
Validation binary_cross_entropy = 0.704920
Epoch 12
Validation binary_cross_entropy = 0.809647
Epoch 13
Validation binary_cross_entropy = 0.725389
Epoch 14
Loss = 2.1635e-01, PNorm = 68.5048, GNorm = 4.7930, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.666692
Epoch 15
Validation binary_cross_entropy = 0.630613
Epoch 16
Validation binary_cross_entropy = 0.616775
Epoch 17
Validation binary_cross_entropy = 0.716855
Epoch 18
Validation binary_cross_entropy = 0.674795
Epoch 19
Loss = 2.4989e-01, PNorm = 68.6414, GNorm = 7.6012, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.637573
Epoch 20
Validation binary_cross_entropy = 0.556882
Epoch 21
Validation binary_cross_entropy = 0.558584
Epoch 22
Validation binary_cross_entropy = 0.708560
Epoch 23
Validation binary_cross_entropy = 0.578200
Epoch 24
Loss = 1.2273e-01, PNorm = 68.7444, GNorm = 6.3779, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.673428
Epoch 25
Validation binary_cross_entropy = 0.620025
Epoch 26
Validation binary_cross_entropy = 0.636128
Epoch 27
Validation binary_cross_entropy = 0.616765
Epoch 28
Validation binary_cross_entropy = 0.604925
Epoch 29
Loss = 1.7007e-01, PNorm = 68.8315, GNorm = 2.8781, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.603113
Epoch 30
Validation binary_cross_entropy = 0.658951
Epoch 31
Validation binary_cross_entropy = 0.612103
Epoch 32
Validation binary_cross_entropy = 0.622953
Epoch 33
Validation binary_cross_entropy = 0.644495
Epoch 34
Loss = 1.4163e-01, PNorm = 68.9081, GNorm = 2.4987, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.642073
Epoch 35
Validation binary_cross_entropy = 0.584537
Epoch 36
Validation binary_cross_entropy = 0.572722
Epoch 37
Validation binary_cross_entropy = 0.604999
Epoch 38
Validation binary_cross_entropy = 0.687661
Epoch 39
Loss = 1.6383e-01, PNorm = 69.0027, GNorm = 1.1458, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.620770
Epoch 40
Validation binary_cross_entropy = 0.649480
Epoch 41
Validation binary_cross_entropy = 0.686936
Epoch 42
Validation binary_cross_entropy = 0.695973
Epoch 43
Validation binary_cross_entropy = 0.697962
Epoch 44
Loss = 1.1958e-01, PNorm = 69.0952, GNorm = 2.5558, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.763982
Epoch 45
Validation binary_cross_entropy = 0.720417
Epoch 46
Validation binary_cross_entropy = 0.783788
Epoch 47
Validation binary_cross_entropy = 0.715663
Epoch 48
Validation binary_cross_entropy = 0.747798
Epoch 49
Loss = 1.2074e-01, PNorm = 69.1757, GNorm = 4.4862, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.774044
Epoch 50
Validation binary_cross_entropy = 0.680517
Epoch 51
Validation binary_cross_entropy = 0.687298
Epoch 52
Validation binary_cross_entropy = 0.680758
Epoch 53
Validation binary_cross_entropy = 0.859506
Epoch 54
Loss = 3.3785e-01, PNorm = 69.2907, GNorm = 16.2946, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.751583
Epoch 55
Validation binary_cross_entropy = 0.882529
Epoch 56
Validation binary_cross_entropy = 0.866150
Epoch 57
Validation binary_cross_entropy = 0.780281
Epoch 58
Validation binary_cross_entropy = 0.841122
Epoch 59
Loss = 8.4068e-02, PNorm = 69.4003, GNorm = 2.3380, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.769181
Epoch 60
Validation binary_cross_entropy = 0.758436
Epoch 61
Validation binary_cross_entropy = 0.756129
Epoch 62
Validation binary_cross_entropy = 0.790245
Epoch 63
Validation binary_cross_entropy = 0.772761
Epoch 64
Loss = 5.8748e-02, PNorm = 69.4858, GNorm = 2.8360, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.685862
Epoch 65
Validation binary_cross_entropy = 0.687388
Epoch 66
Validation binary_cross_entropy = 0.705306
Epoch 67
Validation binary_cross_entropy = 0.717738
Epoch 68
Validation binary_cross_entropy = 0.685935
Epoch 69
Loss = 5.4837e-02, PNorm = 69.5739, GNorm = 4.5144, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.724110
Epoch 70
Validation binary_cross_entropy = 0.720957
Epoch 71
Validation binary_cross_entropy = 0.740238
Epoch 72
Validation binary_cross_entropy = 0.741088
Epoch 73
Validation binary_cross_entropy = 0.767598
Epoch 74
Loss = 4.2111e-02, PNorm = 69.6452, GNorm = 0.8057, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.839197
Epoch 75
Validation binary_cross_entropy = 0.779125
Epoch 76
Validation binary_cross_entropy = 0.724134
Epoch 77
Validation binary_cross_entropy = 0.725413
Epoch 78
Validation binary_cross_entropy = 0.708146
Epoch 79
Loss = 1.2893e-01, PNorm = 69.6971, GNorm = 4.4393, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.869552
Epoch 80
Validation binary_cross_entropy = 1.003334
Epoch 81
Validation binary_cross_entropy = 0.771580
Epoch 82
Validation binary_cross_entropy = 0.679767
Epoch 83
Validation binary_cross_entropy = 0.676682
Epoch 84
Loss = 6.9264e-02, PNorm = 69.7559, GNorm = 2.6131, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.753677
Epoch 85
Validation binary_cross_entropy = 0.839940
Epoch 86
Validation binary_cross_entropy = 0.781771
Epoch 87
Validation binary_cross_entropy = 0.777153
Epoch 88
Validation binary_cross_entropy = 0.838028
Epoch 89
Loss = 2.2367e-01, PNorm = 69.8342, GNorm = 1.9498, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.797502
Epoch 90
Validation binary_cross_entropy = 0.809709
Epoch 91
Validation binary_cross_entropy = 0.890018
Epoch 92
Validation binary_cross_entropy = 0.895983
Epoch 93
Validation binary_cross_entropy = 0.856117
Epoch 94
Loss = 4.6173e-02, PNorm = 69.9098, GNorm = 2.1009, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.874519
Epoch 95
Validation binary_cross_entropy = 0.858954
Epoch 96
Validation binary_cross_entropy = 0.859336
Epoch 97
Validation binary_cross_entropy = 0.862052
Epoch 98
Validation binary_cross_entropy = 0.821092
Epoch 99
Loss = 3.9133e-02, PNorm = 69.9913, GNorm = 2.0398, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.793081
Epoch 100
Validation binary_cross_entropy = 0.791124
Epoch 101
Validation binary_cross_entropy = 0.804284
Epoch 102
Validation binary_cross_entropy = 0.835692
Epoch 103
Validation binary_cross_entropy = 0.886045
Epoch 104
Loss = 5.9781e-02, PNorm = 70.0867, GNorm = 1.9467, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.872024
Epoch 105
Validation binary_cross_entropy = 0.823456
Epoch 106
Validation binary_cross_entropy = 0.820755
Epoch 107
Validation binary_cross_entropy = 0.823303
Epoch 108
Validation binary_cross_entropy = 0.798973
Epoch 109
Loss = 1.7782e-02, PNorm = 70.1651, GNorm = 1.5261, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.778127
Epoch 110
Validation binary_cross_entropy = 0.788933
Epoch 111
Validation binary_cross_entropy = 0.819588
Epoch 112
Validation binary_cross_entropy = 0.916154
Epoch 113
Validation binary_cross_entropy = 0.884067
Epoch 114
Loss = 8.3247e-03, PNorm = 70.2601, GNorm = 0.3892, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.833976
Epoch 115
Validation binary_cross_entropy = 0.802559
Epoch 116
Validation binary_cross_entropy = 0.845982
Epoch 117
Validation binary_cross_entropy = 0.859361
Epoch 118
Validation binary_cross_entropy = 0.846584
Epoch 119
Loss = 1.0578e-02, PNorm = 70.3206, GNorm = 0.8413, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.810104
Epoch 120
Validation binary_cross_entropy = 0.786185
Epoch 121
Validation binary_cross_entropy = 0.793470
Epoch 122
Validation binary_cross_entropy = 0.798146
Epoch 123
Validation binary_cross_entropy = 0.802531
Epoch 124
Loss = 1.4225e-02, PNorm = 70.3738, GNorm = 2.7847, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.816861
Epoch 125
Validation binary_cross_entropy = 0.842395
Epoch 126
Validation binary_cross_entropy = 0.872863
Epoch 127
Validation binary_cross_entropy = 0.894411
Epoch 128
Validation binary_cross_entropy = 0.888762
Epoch 129
Loss = 3.7466e-03, PNorm = 70.4137, GNorm = 0.1997, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.858874
Epoch 130
Validation binary_cross_entropy = 0.837272
Epoch 131
Validation binary_cross_entropy = 0.861976
Epoch 132
Validation binary_cross_entropy = 0.874717
Epoch 133
Validation binary_cross_entropy = 0.878990
Epoch 134
Loss = 4.9301e-03, PNorm = 70.4429, GNorm = 0.2210, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.872859
Epoch 135
Validation binary_cross_entropy = 0.862573
Epoch 136
Validation binary_cross_entropy = 0.839467
Epoch 137
Validation binary_cross_entropy = 0.840911
Epoch 138
Validation binary_cross_entropy = 0.936287
Epoch 139
Loss = 1.3286e-01, PNorm = 70.4728, GNorm = 6.3504, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.927936
Epoch 140
Validation binary_cross_entropy = 0.840583
Epoch 141
Validation binary_cross_entropy = 0.776047
Epoch 142
Validation binary_cross_entropy = 0.783446
Epoch 143
Validation binary_cross_entropy = 0.855897
Epoch 144
Loss = 1.5669e-02, PNorm = 70.5598, GNorm = 1.2892, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.009385
Epoch 145
Validation binary_cross_entropy = 1.009909
Epoch 146
Validation binary_cross_entropy = 0.950985
Epoch 147
Validation binary_cross_entropy = 0.898290
Epoch 148
Validation binary_cross_entropy = 0.856968
Epoch 149
Loss = 2.0329e-02, PNorm = 70.7989, GNorm = 1.8897, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.845441
Epoch 150
Validation binary_cross_entropy = 0.857781
Epoch 151
Validation binary_cross_entropy = 0.907097
Epoch 152
Validation binary_cross_entropy = 0.975241
Epoch 153
Validation binary_cross_entropy = 0.997205
Epoch 154
Loss = 2.4086e-02, PNorm = 70.9579, GNorm = 1.3851, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.925988
Epoch 155
Validation binary_cross_entropy = 0.867438
Epoch 156
Validation binary_cross_entropy = 0.843431
Epoch 157
Validation binary_cross_entropy = 0.852346
Epoch 158
Validation binary_cross_entropy = 0.892109
Epoch 159
Loss = 5.4206e-03, PNorm = 71.0467, GNorm = 0.3447, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.929827
Epoch 160
Validation binary_cross_entropy = 0.924698
Epoch 161
Validation binary_cross_entropy = 0.880296
Epoch 162
Validation binary_cross_entropy = 0.852084
Epoch 163
Validation binary_cross_entropy = 0.835410
Epoch 164
Loss = 1.2481e-02, PNorm = 71.1014, GNorm = 0.2062, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.852611
Epoch 165
Validation binary_cross_entropy = 0.915441
Epoch 166
Validation binary_cross_entropy = 0.984393
Epoch 167
Validation binary_cross_entropy = 1.024861
Epoch 168
Validation binary_cross_entropy = 1.004487
Epoch 169
Loss = 8.6344e-03, PNorm = 71.1402, GNorm = 0.1461, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.923996
Epoch 170
Validation binary_cross_entropy = 0.866040
Epoch 171
Validation binary_cross_entropy = 0.873826
Epoch 172
Validation binary_cross_entropy = 0.982034
Epoch 173
Validation binary_cross_entropy = 1.082899
Epoch 174
Loss = 1.9397e-02, PNorm = 71.1923, GNorm = 1.0257, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.057467
Epoch 175
Validation binary_cross_entropy = 0.908961
Epoch 176
Validation binary_cross_entropy = 0.830966
Epoch 177
Validation binary_cross_entropy = 0.797083
Epoch 178
Validation binary_cross_entropy = 0.783120
Epoch 179
Loss = 3.5386e-02, PNorm = 71.2545, GNorm = 2.0260, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.789018
Epoch 180
Validation binary_cross_entropy = 0.854981
Epoch 181
Validation binary_cross_entropy = 0.940434
Epoch 182
Validation binary_cross_entropy = 0.948077
Epoch 183
Validation binary_cross_entropy = 0.863024
Epoch 184
Loss = 1.1361e-01, PNorm = 71.3454, GNorm = 5.7425, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.840871
Epoch 185
Validation binary_cross_entropy = 0.976784
Epoch 186
Validation binary_cross_entropy = 1.088500
Epoch 187
Validation binary_cross_entropy = 1.048757
Epoch 188
Validation binary_cross_entropy = 0.950155
Epoch 189
Loss = 1.4598e-03, PNorm = 71.4022, GNorm = 0.0079, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.893221
Epoch 190
Validation binary_cross_entropy = 0.890358
Epoch 191
Validation binary_cross_entropy = 0.891970
Epoch 192
Validation binary_cross_entropy = 0.925565
Epoch 193
Validation binary_cross_entropy = 1.043459
Epoch 194
Loss = 1.2918e-02, PNorm = 71.4911, GNorm = 0.3449, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.121884
Epoch 195
Validation binary_cross_entropy = 1.080868
Epoch 196
Validation binary_cross_entropy = 0.966531
Epoch 197
Validation binary_cross_entropy = 0.906958
Epoch 198
Validation binary_cross_entropy = 0.905927
Epoch 199
Loss = 4.7902e-02, PNorm = 71.5778, GNorm = 2.0788, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.913156
Epoch 200
Validation binary_cross_entropy = 0.985414
Epoch 201
Validation binary_cross_entropy = 1.048321
Epoch 202
Validation binary_cross_entropy = 1.075970
Epoch 203
Validation binary_cross_entropy = 1.027761
Epoch 204
Loss = 1.1949e-02, PNorm = 71.6608, GNorm = 0.4287, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.961737
Epoch 205
Validation binary_cross_entropy = 0.925839
Epoch 206
Validation binary_cross_entropy = 0.907770
Epoch 207
Validation binary_cross_entropy = 0.901202
Epoch 208
Validation binary_cross_entropy = 0.921232
Epoch 209
Loss = 1.9811e-02, PNorm = 71.7336, GNorm = 1.9976, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.933060
Epoch 210
Validation binary_cross_entropy = 0.932192
Epoch 211
Validation binary_cross_entropy = 0.932173
Epoch 212
Validation binary_cross_entropy = 0.936548
Epoch 213
Validation binary_cross_entropy = 0.936818
Epoch 214
Loss = 2.0671e-02, PNorm = 71.7726, GNorm = 0.4942, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.907533
Epoch 215
Validation binary_cross_entropy = 0.929456
Epoch 216
Validation binary_cross_entropy = 0.967854
Epoch 217
Validation binary_cross_entropy = 1.022683
Epoch 218
Validation binary_cross_entropy = 1.069332
Epoch 219
Loss = 1.4924e-03, PNorm = 71.8559, GNorm = 0.1043, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.103491
Epoch 220
Validation binary_cross_entropy = 1.126191
Epoch 221
Validation binary_cross_entropy = 1.157878
Epoch 222
Validation binary_cross_entropy = 1.163992
Epoch 223
Validation binary_cross_entropy = 1.127503
Epoch 224
Loss = 3.2944e-03, PNorm = 71.9007, GNorm = 0.1755, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.081326
Epoch 225
Validation binary_cross_entropy = 1.049570
Epoch 226
Validation binary_cross_entropy = 1.042949
Epoch 227
Validation binary_cross_entropy = 1.103632
Epoch 228
Validation binary_cross_entropy = 1.138996
Epoch 229
Loss = 4.0686e-02, PNorm = 71.9631, GNorm = 1.3319, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.106974
Epoch 230
Validation binary_cross_entropy = 1.097390
Epoch 231
Validation binary_cross_entropy = 1.100706
Epoch 232
Validation binary_cross_entropy = 1.107270
Epoch 233
Validation binary_cross_entropy = 1.108851
Epoch 234
Loss = 4.7851e-03, PNorm = 72.0369, GNorm = 0.5329, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.097154
Epoch 235
Validation binary_cross_entropy = 1.078114
Epoch 236
Validation binary_cross_entropy = 1.046345
Epoch 237
Validation binary_cross_entropy = 1.017148
Epoch 238
Validation binary_cross_entropy = 0.998581
Epoch 239
Loss = 2.8359e-03, PNorm = 72.0796, GNorm = 0.2756, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.992001
Epoch 240
Validation binary_cross_entropy = 0.998781
Epoch 241
Validation binary_cross_entropy = 1.040404
Epoch 242
Validation binary_cross_entropy = 1.128735
Epoch 243
Validation binary_cross_entropy = 1.201144
Epoch 244
Loss = 1.2033e-02, PNorm = 72.1642, GNorm = 0.6753, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.207775
Epoch 245
Validation binary_cross_entropy = 1.135404
Epoch 246
Validation binary_cross_entropy = 1.040114
Epoch 247
Validation binary_cross_entropy = 0.977090
Epoch 248
Validation binary_cross_entropy = 0.946183
Epoch 249
Loss = 2.2655e-03, PNorm = 72.2570, GNorm = 0.1447, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.930723
Epoch 250
Validation binary_cross_entropy = 0.929499
Epoch 251
Validation binary_cross_entropy = 0.933863
Epoch 252
Validation binary_cross_entropy = 0.943097
Epoch 253
Validation binary_cross_entropy = 0.979701
Epoch 254
Loss = 3.2165e-03, PNorm = 72.3290, GNorm = 0.0075, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.038946
Epoch 255
Validation binary_cross_entropy = 1.095945
Epoch 256
Validation binary_cross_entropy = 1.134554
Epoch 257
Validation binary_cross_entropy = 1.139673
Epoch 258
Validation binary_cross_entropy = 1.153204
Epoch 259
Loss = 2.8160e-02, PNorm = 72.3560, GNorm = 2.5764, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.138768
Epoch 260
Validation binary_cross_entropy = 1.087834
Epoch 261
Validation binary_cross_entropy = 1.052588
Epoch 262
Validation binary_cross_entropy = 1.032037
Epoch 263
Validation binary_cross_entropy = 1.027824
Epoch 264
Loss = 1.4936e-02, PNorm = 72.3694, GNorm = 0.7837, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.024949
Epoch 265
Validation binary_cross_entropy = 1.000861
Epoch 266
Validation binary_cross_entropy = 0.990947
Epoch 267
Validation binary_cross_entropy = 0.995583
Epoch 268
Validation binary_cross_entropy = 1.002604
Epoch 269
Loss = 2.7326e-02, PNorm = 72.4004, GNorm = 0.1155, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.043444
Epoch 270
Validation binary_cross_entropy = 1.075116
Epoch 271
Validation binary_cross_entropy = 1.111127
Epoch 272
Validation binary_cross_entropy = 1.138786
Epoch 273
Validation binary_cross_entropy = 1.132190
Epoch 274
Loss = 1.9021e-03, PNorm = 72.4347, GNorm = 0.0936, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.121892
Epoch 275
Validation binary_cross_entropy = 1.104308
Epoch 276
Validation binary_cross_entropy = 1.082125
Epoch 277
Validation binary_cross_entropy = 1.058656
Epoch 278
Validation binary_cross_entropy = 1.037936
Epoch 279
Loss = 1.3805e-03, PNorm = 72.4526, GNorm = 0.0113, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.025250
Epoch 280
Validation binary_cross_entropy = 1.014986
Epoch 281
Validation binary_cross_entropy = 1.005821
Epoch 282
Validation binary_cross_entropy = 1.000079
Epoch 283
Validation binary_cross_entropy = 1.006384
Epoch 284
Loss = 5.5809e-04, PNorm = 72.4648, GNorm = 0.0574, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.017240
Epoch 285
Validation binary_cross_entropy = 1.031460
Epoch 286
Validation binary_cross_entropy = 1.044993
Epoch 287
Validation binary_cross_entropy = 1.061453
Epoch 288
Validation binary_cross_entropy = 1.081384
Epoch 289
Loss = 8.3513e-04, PNorm = 72.4784, GNorm = 0.0491, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.099377
Epoch 290
Validation binary_cross_entropy = 1.113964
Epoch 291
Validation binary_cross_entropy = 1.122179
Epoch 292
Validation binary_cross_entropy = 1.120525
Epoch 293
Validation binary_cross_entropy = 1.112329
Epoch 294
Loss = 2.0249e-03, PNorm = 72.4850, GNorm = 0.2165, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.097968
Epoch 295
Validation binary_cross_entropy = 1.074527
Epoch 296
Validation binary_cross_entropy = 1.053458
Epoch 297
Validation binary_cross_entropy = 1.038451
Epoch 298
Validation binary_cross_entropy = 1.025231
Epoch 299
Loss = 9.3040e-04, PNorm = 72.4889, GNorm = 0.0666, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.015851
Model 0 best validation binary_cross_entropy = 0.529347 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.363585
Ensemble test binary_cross_entropy = 0.363585
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.202354
	Seed 1 ==> test binary_cross_entropy = 0.178040
	Seed 2 ==> test binary_cross_entropy = 0.188028
	Seed 3 ==> test binary_cross_entropy = 0.311761
	Seed 4 ==> test binary_cross_entropy = 0.223997
	Seed 5 ==> test binary_cross_entropy = 0.182930
	Seed 6 ==> test binary_cross_entropy = 0.249590
	Seed 7 ==> test binary_cross_entropy = 0.216313
	Seed 8 ==> test binary_cross_entropy = 0.208128
	Seed 9 ==> test binary_cross_entropy = 0.363585
Overall test binary_cross_entropy = 0.232473 +/- 0.057442
Elapsed time = 0:05:36
